{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#线性回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本章介绍用线性模型处理回归问题。从简单问题开始，先处理一个响应变量和一个解释变量的一元问题。然后，我们介绍多元线性回归问题（multiple linear regression），线性约束由多个解释变量构成。紧接着，我们介绍多项式回归分析（polynomial regression问题），一种具有非线性关系的多元线性回归问题。最后，我们介绍如果训练模型获取目标函数最小化的参数值。在研究一个大数据集问题之前，我们先从一个小问题开始学习建立模型和学习算法。\n",
    "\n",
    "<!-- TEASER_END-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##一元线性回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上一章我们介绍过在监督学习问题中用训练数据来估计模型参数。训练数据由解释变量的历史观测值和对应的响应变量构成。模型可以预测不在训练数据中的解释变量对应的响应变量的值。回归问题的目标是预测出响应变量的连续值。本章我们将学习一些线性回归模型，后面会介绍训练数据，建模和学习算法，以及对每个方法的效果评估。首先，我们从简单的一元线性回归问题开始。\n",
    "\n",
    "假设你想计算匹萨的价格。虽然看看菜单就知道了，不过也可以用机器学习方法建一个线性回归模型，通过分析匹萨的直径与价格的数据的线性关系，来预测任意直径匹萨的价格。我们先用scikit-learn写出回归模型，然后我们介绍模型的用法，以及将模型应用到具体问题中。假设我们查到了部分匹萨的直径与价格的数据，这就构成了训练数据，如下表所示：\n",
    "\n",
    "|训练样本|直径（英寸）|价格（美元）|\n",
    "|::|::|::|\n",
    "|1|6|7|\n",
    "|2|8|9|\n",
    "|3|10|13|\n",
    "|4|14|17.5|\n",
    "|5|18|18|\n",
    "\n",
    "我们可以用matplotlib画出图形："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.font_manager import FontProperties\n",
    "font = FontProperties(fname=r\"c:\\windows\\fonts\\msyh.ttc\", size=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEaCAYAAADzDTuZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHERJREFUeJzt3XvYXGV57/HvT0CBgBLOAQKBCqIVBFS2FdwEq5BsEEUU\n5NCCgA2bM1qFYMth61ZLFakbQbhAEhAQKsiphWJrsICCgRJAObhbkiolhIOJBlEC5O4fzzM8k8l7\nmpnMrJl5f5/ryvXOWmtmree9r8m633Xf66CIwMzMxrfXVT0AMzOrnpOBmZk5GZiZmZOBmZnhZGBm\nZjgZmJkZTgZmK5DUtf8Tks6WtFYLn1u9E+Ox8c1fKusJktYF7gRGuvDlj4G78+udgQeGeP14RBwj\naU3gmoj4cF7/scCngPdHxOK67R4GrBYRs/OO+V5JXwRezW+5KyIW5ffeHxHvHOF32BrYrG7WAcBS\n4Pa6ec9HxGOSDgU2Bt4u6Zy65RsAkyJiI0nbALsCE4AlwGTgeuAMSX8XEQ/XbfuLwM3AdGDfvN2a\nScCpEXHjcGM3czKwXrEasDgi9pS0J7BXRMyUNDci3t34Zkmfi4hz8uvzI+L4hrfsD/wwL58GbA8c\nDFwu6dCI+K2kC4D3p7doM9JO92fAXcCbgKuBvfM6VgO2lfSTYcb/IHARsEVtiHkda9TNq419GnAs\nsG9OTHvm+RuQdugfz2+dT0ooi4F1gTUj4peS/gSYI2lHUpKYCOxGSiRrAjMi4v667R01zJjNXuNk\nYL3oDcDLjTMlbQucRDp6eLBu0fqStoiIJ+vmHQv8taTPA5sDx0fEcklnAjdLuiQijpX0f4Cn8vp2\nyZ/9OOkv7E/XjgryOv41IvYdaeCSdgGOBl4gJZcgJaK1gasj4hpJfwvsB3xL0hkR8bik9wJXAldF\nxB15dTOBD5KOfJ4EnpW0BHg8r/944Arg0fw7PAq8GfimpPojgw2Az480bjMnA+tFb2TFMkfN5qS/\nkJ8EtpN0PvAe4NfAVZLeCHwVeAXYBliHtINcCOxA2uH/GVA7UgD4MGmH/bfABcChwJeB/wKWSVKk\ne7ZsBLxH0txhxrxbRCzL25wD3EM6OqjZkXSkQER8VtLewG+BCZKuBNYilXdOlHQLKel9hfR/dI38\n8yvAacCiiLhR0onADfko5wngXtJRwnERcb+kU4HbIqI+cZoNycnAetH2wEP1M3JpZQfgEeBPSTvt\nJcClwAzgLGBORNwm6VzgMlLZ6e5cT/9FXtVOwEsR8YCk9wO/AS4BtgX+BbiV9Nf1VsBfAqcAB+Sy\ny4ZDDTb3EpbVzXqV0nOAlBRezT+RNBmYBRwGHAmcHxG18tMMSVOBo0hJ7V7gOeA/gD2ATwN/l5vI\na+dEsCXwNuAjpOS3j6TfAVOAj+ejhMcj4pihxm8GTgbWYyRtTKr3fzXPejnX0nci1c7XIyWAI0kl\nlABuyu/dRdL7gM8AZ9StdiJpp9/oSOAa0k76QlJDd7/8r+aAZn8F4EOkI5Z66wPX5dffAX5C2tnP\nAb4kCVKjd2F+z+eAQ/LvsTXwc2BpRJwu6RHga3kdAKeTEtVJpBLXpyPiyVwSu6W+f2A2HCcD6xVB\nOtX5CuCkiHgxzz8fuIX01/EsUqloEWmHvTwi9hpqZXnnWjsz6R2UI4N6nyU1kFePiEWS3hoR76tb\nx2XAWpKuJ53JM9SYz2bFclCQdvp3N8zfmdQLAfgYqadBRFyX34+kWyNiet32HyOdYbV13v6pedEF\nwDzgLXkdx0j6AvC7/L4Zkj5IKkPtno8c/iIifjTE72AGOBlY7xCwHJgeEctrMyPiKuAqgFw+eT1p\nB3495aybYdeZP/NURLzUuDAiFuakMaKI+OgIG3gL8HTD7zGBdDQiUnIQqdexLK/v2TFud6mkmaTy\n1UbAXvk6iFNJPYlTJB2XexqvI/Uk/kA64jiudkQg6TOjbszGvY4ng1wfvZx0CB7AxRHxDUlnkc66\neDa/dWZE3Nbp8VjPeoVUArp/hB3lmqRewNdJp2AePYb1ngocVze9jJWvZXhtWtKddfPfDJzZuEJJ\nZwMHAS+RzhI6Ps+vnaq6jHSqZ711SM3iHSPioLp1zal7z4S66fmk/sUhwI/zv5OA7YAfR8Shki4E\nvifpP0klMwHfIJ2qWn9G0WRguMa3GQDq9MNtJG0KbBoR8yStA9xPanQdSKqBntvRAZj1KUnvBh6L\niKV183aNiJ/WTW8ZEb/Mr2tnPpk1reNHBhHxNPkwOiJekPQoqe4LK9ZUzaxORKz013x9IsjTv6x7\n7URgLevqvYkkTSE10u7Js06Q9KCkSyWt182xmJlZ0c2bcq0DfI90psgLpFP5tiadMriQdKqcmZlV\noOM9AwBJa5BOD7w1Is4bYvkU4OaI2KFhvg97zcxaEBFNleG7cTaRSBcJPVKfCCRNiojaBTb7Aw8P\n9flmf6FBJemsiDir6nH0AseicCwKx6Jo5Q/pblxnsBvpsvuHJNVuM3w6cLCknUin9c0n3VLAhjel\n6gH0kClVD6CHTKl6AD1kStUD6GfdOJvoLobuTdza6W2bmdnY+Eln/WNW1QPoIbOqHkAPmVX1AHrI\nrKoH0M+60kBuVb6Gxj0DM7MmtLLv9JFBn8j32DEci3qOReFYtMfJwMzMXCYyMxs0LhOZmVlLnAz6\nhOuhhWNROBaFY9EeJwMzM3PPwMxs0LhnYGZmLXEy6BOuhxaOReFYFI5Fe5wMzMzMPQMzs0HjnoGZ\nmbXEyaBPuB5aOBaFY1E4Fu1xMjAzM/cMzMwGjXsGZmbWEieDPuF6aOFYFI5F4Vi0x8nAzMzcMzAz\nGzTuGZiZWUucDPqE66GFY1E4FoVj0R4nAzMzc8/AzGzQuGdgZmYtcTLoE66HFo5F4VgUjkV7nAzM\nzMw9AzOzQeOegZmZtcTJoE+4Hlo4FoVjUTgW7XEyMDMz9wzMzAaNewZmZtYSJ4M+4Xpo4VgUjkXh\nWLTHycDMzDrfM5A0Gbgc2BgI4OKI+Iak9YFrgK2ABcCBEbGk4bPuGZiZNamVfWc3ksGmwKYRMU/S\nOsD9wEeATwLPRcQ5kk4FJkbEaQ2fdTIwM2tSTzaQI+LpiJiXX78APApsDuwHzM5vm01KEDYM10ML\nx6JwLArHoj1d7RlImgLsDNwLbBIRi/KiRcAm3RyLmZkVq3drQ7lEdB1wUkQslcoRTESEpCHrVZJm\nkXoKAEuAeRFxR142NX9+4Kcj4o5eGo+ne2e6plfGU9V0bV6vjKeb0/n1ETkMC2hBVy46k7QGcAtw\na0Scl+c9BkyNiKclTQLmRMT2DZ9zz8DMrEk92TNQOgS4FHiklgiym4DD8+vDgRs6PZZ+5npo4VgU\njkXhWLSnG2Wi3YDDgIckPZDnzQS+Alwr6SjyqaVdGIuZmQ3B9yYyMxswPVkmMjOz3udk0CdcDy0c\ni8KxKByL9jgZmJmZewZmZoPGPQMzM2uJk0GfcD20cCwKx6JwLNrjZGBmZu4ZmJkNGvcMzMysJU4G\nfcL10MKxKByLwrFoj5OBmZm5Z2BmNmha2Xd27eE2Zjb+SLoY2A54ETgkIpZUPCQbhstEfcL10MKx\nKPogFtsBewDTgYs6uaE+iEVPczIws056Mf+cC8yociA2MvcMzKxjJK1HOiKY4RJR97Sy73QyMDMb\nML7obIC5HloMYiwkXSzpDkn/mP+aHuvnpnZwWH3FsWiPk4FZb+hao9VsKC4TmfUASf9ISgRzgb1c\nX7d2uGdg1qfcaLVVyT2DAeZ6aDGIsYiIJRFxULOJYBBj0SrHoj1OBmZm5jKRmdmgcZnIzMxa4mTQ\nJ1wPLRyLwrEoHIv2OBmYmZl7BmZmg8Y9AzMza4mTQZ9wPbRwLArHonAs2uNkYGZm7hmYmQ0a9wzM\nzKwlTgZ9wvXQwrEoHIvCsWiPk4GZmXWnZyDp28A+wDMRsUOedxZwNPBsftvMiLit4XPuGZiZNamX\newaXAdMa5gVwbkTsnP/dNsTnzMysC0ZNBpJWk/Tn+dmsj+Z/P5N0maT3jGUjEXEnsHio1Tc74PHK\n9dDCsSgci8KxaM+IyUDSJsA/AFsAx0XEWyPircA7gG8BfyHpgja2f4KkByVd2sxDwM3MbNUasWcg\naRvg9xGxcIT37BERPxp1Q9IU4Oa6nsHGlH7BF4BJEXFUw2fcMzAza1Ir+87VR1oYEU+MtoKxJIJh\nPvdM7bWkS4Cbh3qfpFnAgjy5BJgXEXfkZVPzujztaU97etxO59dHkCygBV27AnmII4NJtSMOSacA\n746IQxo+4yODTNLU2pdgvHMsCseicCyKVX5ksKpIuhrYA9hQ0q+AM4GpknYinVU0H5jRjbGYmdnK\nxnxkIOmbEXFcfn0O8G7gkoi4smOD85GBmVnTWtl3jtZA/k5EHJZf/yQi/qRu2dbAbRHxllYHPOrg\nnAzMzJrWiYvO/mi4BRExH3iumY1Z63wOdeFYFI5F4Vi0p5mewZqS/r7hsxNW8XjMzKwCo5WJXisN\nSboHOJxy1XAAv4yI33dscC4TmZk1rdNnE0VEPN7kmMzMrA80c6O6tZXuSfRjSefl6wasS1wPLRyL\nwrEoHIv2NJMMXoyItwMfAR4Cbpe0S2eGZWZm3TRaz6D+2oLGU0t3BK4CdomIZR0ZnHsGZmZNW+Wn\nltYSQfajhmUPARcAGzazQTMz6z1jeZ7BX0paHThD0vl53hfy4ieBSR0cn2WuhxaOReFYFI5Fe8Zy\nNtE+wBPAq8Af53m75597Ajd1YFxmZtZFwx4ZSNpe0r61SVjxqWSSBOwK3NW54VmN78ZYOBaFY1E4\nFu0Z6cjgV8CFwBqkC8waO80nA7Mj4uUOjc3MzLpk2CODiPgdMB14U/75v4CNJH0C2BnYICIu7soo\nzfXQTNLFkuYpPZN73D8q1d+LwrFoz2hnE/0BWA78AXgpv36RdESxhaRmrlMwWxW2Iz2DezpwUcVj\nMRsYI/UM1pJ0I7AYuAP4IfB8RNwEzAV+THp2sXWB66GveTH/nIsfiOTvRR3Hoj0j/WX/duDrlH7B\nCj2DXCL6n5Le1Lnhma3kEOBaYK+IWFL1YMwGxUg9g7l1mXZ5/tfoFuCDHRiXNXA9NMkJ4EIngsTf\ni8KxaM9YrjO4JiJuAJBU2/Hfln/eQWowm5lZHxvzM5Cr4HsTmZk1rxOPvTQzs3HAyaBPuB5aOBaF\nY1E4Fu0ZMRlI+iNJm43ynt1HWm5mZr1vtOcZbArMAu4EroqI+XXL3gUcA7wSEcd0ZHDuGZiZNa2V\nfeeoDeR8++pP5H/bkG5Ytxz4N+CiiOjYjeqcDMzMmteRZFAlJ4NC0lRfYZk4FoVjUTgWhc8mMjOz\nlvjIwMxswPjIwMzMWjLaqaWbSNokv5ak6yRtLGmr/G/L7gzTfA514VgUjkXhWLRntHsT7Q78taRP\nA8uA54HrgH/Py6cBkzo3PDMz64bRrjM4AHiEdCvr9YA/By6NiPfl5XfWXndkcO4ZmJk1rRM9g9WA\n3wC3ApsD/9ni2MzMrIeN9KSz1wF/BXwV2AiYCZzYpXFZA9dDC8eicCwKx6I9Iz3cZjnwP4AngIXA\nlaTnzj4saY6kOcArXRmlmZl11FhuRzGT9BCbbYBtSU+ZWtT5oblnYGbWilb2nSOeTSTpcOAIYDHp\npnTnAtOk17YREXH5GAb2bWAf4JmI2CHPWx+4BtgKWAAc6EcZmplVY7QG8qtAAJuRblB3NLAOqTz0\nSl4+FpeRTkOtdxrwg4jYDviXPG3DcD20cCwKx6JwLNoz4pFBRHxH0nuBX5COECYCRwKfjYiFY91I\nRNwpaUrD7P2APfLr2aRSlBOCmVkFxtIzeCNwM+kv+2V5du1DEWO8uVFOBjfXlYkWR8TE/FrAr2vT\ndZ9xz8DMrEmd6BlMBC4HTgG+BOzU8JZlwN7NbHAoERGSeveOeWZmA26021H8EDg4Ih4jPcxmVVok\nadOIeFrSJOCZod4kaRapwQywBJhXu2d5rUY4Hqbr66G9MJ4qpxtjUvV4Kp7eKSLO66HxVDl9MuN7\n/3AEyQJaMNrtKI4FtgdOBnYA1iVdkfyLiHipqQ2tXCY6B3g+Iv5G0mnAehFxWsNnXCbK5Ad3vMax\nKByLwrEoWtl3jqVnMI3U7L2PdK3BROAdwFPAaRHxxBgGdjWpWbwhsAg4A7gRuBbYkmFOLXUyMDNr\nXkeSQV7x54FHI+L6unlTgVMjYnqzAx3z4JwMzMyatsqTgaQbSX2DB0nXFyzOi4J03QERcXdLox3L\n4JwMXuND4MKxKByLwrEoVvnZRMBBwP7AF0glosuAlxve07FkYGZm3THmZyBL+gSwK/CZGOuH2uQj\nAzOz5nWsZ1AVJ4P+IeliYDvgReCQxpMBzKx7Wtl3jnZvIusRfXDfle1IZ4xNBy7q5Ib6IBZd41gU\njkV7nAxsVXkx/5wLzKhyIGbWPJeJbJWQtB7piGCGS0Rm1XLPwMzM3DMYZK6HFo5F4VgUjkV7nAzM\nzMxlIjOzQeMykZmZtcTJoE+4Hlo4FoVjUTgW7XEyMDMz9wzMzAaNewZmZtYSJ4M+4Xpo4VgUjkXh\nWLTHycDMzNwzMDMbNO4ZmJlZS5wM+oTroYVjUTgWhWPRntGegWx9zE8fM7Oxcs9ggEm6g/T0MYBr\nI+KgCodjZl3inoE18tPHzGxMnAz6RIv10EOAa4G9BqlE5Npw4VgUjkV73DMYYDkBuDRkZqNyz8DM\nbMC4Z2BmZi1xMugTrocWjkXhWBSORXucDMzMzD0DM7NB456BmZm1xMmgT7geWjgWhWNROBbtcTIw\nMzP3DMzMBo17BmZm1pLKk4GkBZIekvSApJ9WPZ5e5Xpo4VgUjkXhWLSnF+5NFMDUiPh11QMxMxuv\nKu8ZSJoPvCsinh9imXsGZmZN6teeQQD/LOk+SZ+qejBmZuNRL5SJdouIhZI2An4g6bGIuLO2UNIs\nYEGeXALMi4g78rKpAONhur4e2gvjqXK6MSZVj6fi6Z0i4rweGk+V0yczvvcPR5AsoAWVl4nqSToT\neCEivpanXSbKJE2tfQnGO8eicCwKx6JoZd9ZaTKQtDawWkQslTQBuB04OyJuz8udDMzMmtTKvrPq\nMtEmwPcl1cZyZS0RmJlZ9/RUmaiRjwwKHwIXjkXhWBSORdGvZxOZmVnFfGRgZjZgfGRgZmYtcTLo\nE77vSuFYFI5F4Vi0x8nAzMzcMzAzGzTuGZiZWUucDPqE66GFY1E4FoVj0R4nAzMzc8/AzGzQuGdg\nZmYtcTLoE66HFo5F4VgUjkV7nAzMzMw9AzOzQeOegZmZtcTJoE+4Hlo4FoVjUTgW7XEyMDMz9wzM\nzAaNewZmZtYSJ4M+4Xpo4VgUjkXhWLTHycDMzNwzMDMbNO4ZmJlZS5wM+oTroYVjUTgWhWPRHicD\nMzNzz8DMbNC4Z2BmZi1xMugTrocWjkXhWBSORXucDMzMzD0DM7NB456BmZm1xMmgT7geWjgWhWNR\nOBbtcTIwMzP3DMzMBo17BmZm1pJKk4GkaZIek/T/JZ1a5Vh6neuhhWNROBaFY9GeypKBpNWA84Fp\nwNuAgyW9tarx9IGdqh5AD3EsCseicCzaUOWRwa7Av0fEgoh4Gfgu8OEKx9Pr1qt6AD3EsSgci8Kx\naEOVyWBz4Fd100/meWZm1mVVJoPePY2pN02pegA9ZErVA+ghU6oeQA+ZUvUA+tnqFW77v4DJddOT\nSUcHK5DkpJFJOrzqMfQKx6JwLArHonWVXWcgaXXgceBPgaeAnwIHR8SjlQzIzGwcq+zIICJekXQ8\n8E/AasClTgRmZtXo6SuQzcysO3r2CmRfkFZIWiDpIUkPSPpp1ePpJknflrRI0sN189aX9ANJv5B0\nu6RxcUrhMLE4S9KT+bvxgKRpVY6xWyRNljRH0s8l/UzSiXn+uPtujBCLpr4bPXlkkC9Iexz4AKnR\nPJdx3E+QNB94Z0T8uuqxdJuk9wEvAJdHxA553jnAcxFxTv5DYWJEnFblOLthmFicCSyNiHMrHVyX\nSdoU2DQi5klaB7gf+AjwScbZd2OEWBxIE9+NXj0y8AVpKxuXN+yLiDuBxQ2z9wNm59ezSV/8gTdM\nLGAcfjci4umImJdfvwA8SrpOadx9N0aIBTTx3ejVZOAL0lYUwD9Luk/Sp6oeTA/YJCIW5deLgE2q\nHEwPOEHSg5IuHQ9lkUaSpgA7A/cyzr8bdbG4J88a83ejV5NB79WuqrVbROwMTAeOy+UCAyLVOcfz\n9+VCYGvSfXkWAl+rdjjdlcsi1wEnRcTS+mXj7buRY/E9UixeoMnvRq8mgzFdkDZeRMTC/PNZ4Puk\nMtp4tijXSZE0CXim4vFUJiKeiQy4hHH03ZC0BikRXBERN+TZ4/K7UReL79Ri0ex3o1eTwX3AtpKm\nSHo9cBBwU8VjqoSktSWtm19PAPYCHh75UwPvJqB2penhwA0jvHeg5R1ezf6Mk++GJAGXAo9ExHl1\ni8bdd2O4WDT73ejJs4kAJE0HzqNckPbliodUCUlbk44GIF0keOV4ioWkq4E9gA1JNeAzgBuBa4Et\ngQXAgRGxpKoxdssQsTgTmEoqAwQwH5hRVzMfWJJ2B/4VeIhSCppJupPBuPpuDBOL04GDaeK70bPJ\nwMzMuqdXy0RmZtZFTgZmZuZkYGZmTgZmZoaTgZmZ4WRgZmY4Gdg4JelsSWtVPQ6zXlHlM5DNVql8\ngd5mdbMOAJYCt9fNex54J7Ax8PZ8O+yaDYBJEbHRMOvfFJgQEf8xxLI3A38VEUfk6a8Cfx8R944y\n5o+Rbk8+c4hl2wC/Gw8XkVn1nAxskKwHbJFfC3gTsEbdPIA9gWnAhyJicZ5G0gbAzcDHh1pxfmb3\nLODIhvlfAXYD1gQmS7ozL5oCfEDSUuAx0hXDew+x6onA2pI+ANwXEf+7btnvgdmS9omIV0f75c3a\n4SuQbaBIOgo4mvQQmAmkS/FfBNYmPRdjMvA3wAXAGRHxuKT3AlcCV0XE54dZ72HAVhHxf4dZvjHw\nUdK95NcH1gLubTyKkPT6iFhWN30A6cjg9GHWezrwZERcPsYQmLXERwY2aNYB5pDu517/YI8dgTdG\nxOck7Q38Fpgg6UrSjntf4ERJt5BuAdxYCvoEcMJQG5T0RWAfYAmp/LQ68BRwiqR7IuKE/L69SLcg\n3z8iltevYoTf52rgm4CTgXWUG8g2iF7N/17J/2rTSJpMKvd8l1TyOT8iPhoRP4+IGaR7vh81xDq3\niYj5I2zz5IjYE/gy8PX8eoUHEUXE7aSHNl001l8kb3Prsb7frFU+MrBBI+BDwHsa5q8PXA9cAfyE\nlCTmAF9KdwBmEukBIACfHWa9K8+U/h/pUYv7SfoN+clakg4llak2kbReRPxZ/shJwC2S3hYRj7T0\nG5p1gJOBDZogPeTjblbcge8MvAH4GHAcQERcl9+LpFsjYvoI610+1MxcAqqVgbYklXMWAydExEoP\nZMqN4JG2swJJq5GPasw6ycnABo1If5FPzK8j/1wXWBYRz+UjgWb9m6R3RcR9K20wPXToSNL94w8H\nXgKukHQN8N1R7qe/OanZPZx3AXNbGbBZM5wMbGBIugB4P7CMdLpnvXVIDeN3AD/L759Tt3xC3fT8\niDiy4fMXAceQzlSq3+apwIHAbGCPiHg5z98b+CTw/dxEnln3mc8AnyOd5fQSqaw1nBk00WMwa5VP\nLTUbI0nfIj1j9q66eW+IiJc6tL09gIMi4thOrN+snpOBmZn51FIzM3MyMDMznAzMzAwnAzMzw8nA\nzMxwMjAzM5wMzMwM+G/SFyQahx5UmgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x79c65c0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def runplt():\n",
    "    plt.figure()\n",
    "    plt.title('匹萨价格与直径数据',fontproperties=font)\n",
    "    plt.xlabel('直径（英寸）',fontproperties=font)\n",
    "    plt.ylabel('价格（美元）',fontproperties=font)\n",
    "    plt.axis([0, 25, 0, 25])\n",
    "    plt.grid(True)\n",
    "    return plt\n",
    "\n",
    "plt = runplt()\n",
    "X = [[6], [8], [10], [14], [18]]\n",
    "y = [[7], [9], [13], [17.5], [18]]\n",
    "plt.plot(X, y, 'k.')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上图中，'x'轴表示匹萨直径，'y'轴表示匹萨价格。能够看出，匹萨价格与其直径正相关，这与我们的日常经验也比较吻合，自然是越大越贵。下面我们就用scikit-learn来构建模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "预测一张12英寸匹萨价格：$13.68\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "# 创建并拟合模型\n",
    "model = LinearRegression()\n",
    "model.fit(X, y)\n",
    "print('预测一张12英寸匹萨价格：$%.2f' % model.predict([12])[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一元线性回归假设解释变量和响应变量之间存在线性关系；这个线性模型所构成的空间是一个超平面（hyperplane）。超平面是n维欧氏空间中余维度等于一的线性子空间，如平面中的直线、空间中的平面等，总比包含它的空间少一维。在一元线性回归中，一个维度是响应变量，另一个维度是解释变量，总共两维。因此，其超平面只有一维，就是一条线。\n",
    "\n",
    "上述代码中`sklearn.linear_model.LinearRegression`类是一个估计器（estimator）。估计器依据观测值来预测结果。在scikit-learn里面，所有的估计器都带有`fit()`和`predict()`方法。`fit()`用来分析模型参数，`predict()`是通过`fit()`算出的模型参数构成的模型，对解释变量进行预测获得的值。因为所有的估计器都有这两种方法，所有scikit-learn很容易实验不同的模型。\n",
    "\n",
    "`LinearRegression`类的`fit()`方法学习下面的一元线性回归模型：\n",
    "\n",
    "$$y=\\alpha+\\beta x$$\n",
    "\n",
    "$y$表示响应变量的预测值，本例指匹萨价格预测值，$x$是解释变量，本例指匹萨直径。截距$\\alpha$和相关系数$\\beta$是线性回归模型最关心的事情。下图中的直线就是匹萨直径与价格的线性关系。用这个模型，你可以计算不同直径的价格，8英寸\\$7.33，20英寸\\$18.75。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEaCAYAAADzDTuZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYXVW5x/Hvj9BD76EGFBXpSBQpEiwURa4KgiiKAhqk\nShECKputYolS5FKkN2le4SqoKOpFBSwUCRFpVyUKl6oSpQiB5L1/rD3ZJ8PMZGbOnLP3Oef3eZ48\nc+re77wczjtrrb3WUkRgZma9baGqAzAzs+q5GJiZmYuBmZm5GJiZGS4GZmaGi4GZmeFiYDYfSW37\nf0JSLmmJUbxv4VbEY73NHyqrBUlLAzcDQ0182RC4tbi9OXDXALcfiIgDJS0OfDsidiuOfxDwceCt\nEfF0w3n3AcZFxCWSlgR+K+kLwJziJbdExBPFa++MiDcM8TusC6ze8NDuwDPAjQ2P/T0i7pf0QWAV\nYCNJ0xqeXxGYEBErS1oPeCMwHpgFrAZcB2SSvhERv2849xeB64FdgF2L8/aZABwbEd8bLHYzFwOr\ni3HA0xGxg6QdgB0j4jhJt0fEpP4vlnRMREwrbp8REYf0e8kewM+K53cGXgfsDVwq6UMR8S9JZwFv\nTS/R6sDywO+BW4BlgSuBnYpjjAPWl/TrQeK/GzgHWLMvxOIYizQ81hf7jsDBwK5FYdqheHxF0hf6\n+4uXPkQqKE8DSwNLRcTDkt4M3CRpE1KRWB7YhlRIFgemRMSdDefbf5CYzeZxMbA6Wgx4qf+DktYH\nDie1Hu5ueGoFSWtGxCPF6wQcChwr6TPAGsAhETFXUgZcL+n8iDioaAX8HzCD1MKYTfoy3gU4sq9V\nUBzjlxGx61CBS9oCOAB4lvQXfZAK0ZLAlRFxtaSvAbsB35R0QkQ8IGlr4HLgioj4eXG444B3FHE9\nAjwl6Z/AA8XxDwEuA+4DHi1+vho4U1Jjy2BF4DNDxW3mYmB1tAzzd3P0WYP0F/IjwGsknQFsBfwD\nuELSMsDJlH+NL0X6gnwM2JhUQD4M9LUUAN5D6hL6GnAW8CHgy6QCMVuSIq3ZsjKwlaTbB4l5m4iY\nXZzzJuA3pNZBn01ILQUi4tOS3gH8Cxgv6XJgCVL3zmGSvk8qel8h/T+6SPHzK8BU4ImI+J6kw4Dv\nFq2cPwO/JbUSDo6IOyUdC/woIhoLp9mAXAysjl5H+kt9nqKrZ2PgXuBtpC/tWcAFwBTgROCmiPiR\npFOBi0jdTrcW/ekPFofaDHgxIu4qjvk0cD6wPqlb6QbSX9frAEcDRwC7F90uKw0UbDGWMLvhoTmU\nYw6QisKc4idFl9TFwEeA/YAzIqKv+2mKpMnA/sDLpC/4vwF/ArYHjgS+UQwiL1kUgjWB15MK233A\nuyQ9B0wE3l+0Eh6IiAMHit8MXAysZiStArwX+Hrx0EtFX/pmpC/u5UgFYD9SF0qQBlUBtpC0HekL\n84SGwy4P/HOA030EuJr0JX02aUB3t+Jfn91H+isA7ya1WBqtAFxT3P4W6Uv+ZVIr4kupZ4sJpFYM\nwDHAB4vfY13gD8AzEXG8pHtJLaC+AvJZUqE6nNTFdWREPFJ0iX2/cfzAbDAuBlYXQbrU+TLg8Ih4\nvnj8DOD7pL+OLyZ1FT1B+sKeGxE7DnSw4su178qkTSlbBo2OIg0gLxwRT0jaICK2azjGRcASkq4F\n1hok5pz5u4OC9KV/a7/HNyeNhQDsBRwEEBHXFK9H0g0RsUvD+e8nXWG1bnH+Y4unzgKmA68tjnFg\nMfbxXPG6KQ3dUNtKWhv4RET8YoDfwQxwMbD6EDAX2CUi5vY9GBFXAFcAFN0ni5K+wK+lvOpm0GMW\n73k0Il7s/2REPFYUjSFFxPuGOMFrgcf7/R7jSa0RkYqDSGMds4vjPTXM8z4j6ThS99XKwI7FPIhj\nSWMSR0g6uBjTWIg0JvECqcVxcF+LQNJRCzyZ9byWFwNJawGXkprgAZwbEadLOpF01cVTxUuPi4gf\ntToeq62XSV1Adw7xRbk4aSzgVNIlmAcM47jHki7j7DObV85lmHdf0s0Nj78ayPofUFJO+uv+RdJV\nQocUj/ddqjqbdKlno6VIg8WbRMReDce6qeE14xvuP0Qav/gg8Kvi3+HAa4BfRcSHJJ0NfEfSX0hd\nZgJOJ12q2nhF0VrAYAPfZgCo1ZvbSFoNWC0ipktaCriTNNC1J6kP9JSWBmDWoSRNAu6PiGcaHntj\nRNzWcH/tiPhrcbvvyiezEWt5yyAiHqdoRkfEs5LuI/X7wvx9qmbWICJe8dd8YyEo7v+14bYLgY1a\nW9cmkjSRNJD2m+KhQyXdLekCScu1MxYzMyu1c1GupYDvkK4UeZZ0Kd+6pEsGHyNdKmdmZhVo+ZgB\ngKRFSJcH3hARpw3w/ETg+ojYuN/jbvaamY1CRIyoG74dVxOJNEno3sZCIGlCRPRNsHkvaYGwVxjp\nL9StJJ0YESdWHUcdOBcl56LUSblQrnGk2e1TgZOA0yOLOUO/awTHH8Uf0u2YZ7ANsA8wQ1LfMsPH\nA3tL2ox0Wd9DpCUFbHATqw6gRiZWHUCNTKw6gBqZWHUAw6FcGwEXkhYbfFNk8aeKQwLaczXRLQw8\nNnFDq89tZlYXyrUoaSXaQ0h/EJ8fWX2uAPMM5M5xcdUB1MjFVQdQIxdXHUCNXFx1AINRrkmk1sBM\nYPPI0nLrddKWAeTRKubQeMzAzDqSci0JfJ7UVX4EcFU7WgOj+e70Hsgdolhjx3AuGjkXpbrlQrkm\nk/bQWB3YOLK4sk7dQv25m8jMbAwp1zLANNJmRZ+MLK6vOKRhcTeRmdkYUa53At8EfgQcE1nMqiSO\nUXx3umVgZtYk5VoJOA14M/DRyOJ/Kg5pxDxm0CHq1h9aJeei5FyUqsiFckm59iJNmn0S2KQTCwG4\nZWBmNirKtTpp17n1gfdGFr9ZwFtqzWMGZmYjoFwi7cH9ZdL4wEmRvXInvSp5zMDMrIWUa13gPNKu\nfG+PLGZUHNKY8ZhBh3DfcMm5KDkXpVbmQrnGKdfhpO1DbwS26qZCAG4ZmJkNSbk2IK28/DKwdWTx\nYMUhtYTHDMzMBqBciwDHAJ8CTgDOiSzmVhvV8HjMwMxsDCjXFqSF5R4D3hBZudd0t/KYQYdw33DJ\nuSg5F6WxyIVyLaFcXyYtsX8y8M5eKATgloGZGQDKtS1pbGAGafLYExWH1FYeMzCznqZcSwNfAt4H\nHBpZXFtxSE3zEtZmZiOgXDuRlpIYD2zUDYVgtNxN1CEkTY6In1cdRx04FyXnojSSXCjXCsApwGTg\nE5HFjS0MrSO4ZWBmPUW53gfcA/yL1Bro+UIAHjMwsx6hXKsBZwAbAQdEFrdUHFLLeJ6BmVk/xcJy\nHwG+BpwP7BNZvFBtVPXjbqIO4evJS85FybkoDZQL5Vob+CFpFvHOkcXxLgQDczEws66jXAsp18HA\nncDNwBsji99VHFateczAzLqKcr2GNHlsIWD/yOL+ikNqO48ZmFnPUq6FgaOATwOfB86MLOZUG1Xn\ncDdRh3DfcMm5KDkXiXJtyoP8AXg7MCmyON2FYGTcMjCzjqVciwGfBabwJBfxGqZGVuO+7xrzmIGZ\ndSTl2oq0zPQDwMGRxaMVh1QbHjMws66nXOOBLwIfAA4DvuPWQPM8ZtAh3Ddcci5KvZYL5XobaWG5\nlUhLSfxXXyHotVyMNbcMzKz2lGs50gzinYADI4sfVhxS1/GYgZnVmnLtBpwFXAdMjSz+VXFItecx\nAzPrGsq1MnA6sCXwocjiFxWH1NU8ZtAh3B9aci5K3ZgL5ZJyfZA0NvAIsOlwCkE35qKd3DIws9pQ\nrjWBs4F1gHdHFrdXHFLPaPmYgaS1gEuBVYAAzo2I0yWtAFxN+o8+E9gzImb1e6/HDMx6gHItBBwA\nnAT8J/CVyGJ2tVF1rtF8d7ajGKwGrBYR0yUtRVpF8D3Ax4C/RcQ0SccCy0fE1H7vdTEw63LK9SrS\nPgNLkhaWu6fikDreaL47Wz5mEBGPR8T04vazwH3AGsBuwCXFyy4hFQgbhPtDS85FqZNzoVzjlOtI\n4LfA9cDWzRSCTs5FHbR1zEDSRGBz0n/8VSPiieKpJ4BV2xmLmVVHuTYiLTP9PLBVZPHHikPqeW0r\nBkUX0TXA4RHxjFS2YCIiJA3YXyXpYtKYAsAsYHpE/Lx4bnLx/q6/HxE/r1M8vl+f+33qEs+Q9xdn\nYaayDXAIt3IJP+MHMScVgmaP3/dYrX7fNt0vbn+0SMNMRqEtk84kLQJ8H7ghIk4rHrsfmBwRj0ua\nANwUEa/r9z6PGZh1CeWaRGoN/JU0i/iRikPqWrUcM1BqAlwA3NtXCArXAfsWt/cFvtvqWDqZ+0NL\nzkWpE3KhXEsq1zTSuMBXSJeMjnkh6IRc1Fk7uom2AfYBZki6q3jsONKH4tuS9qe4tLQNsZhZGynX\n9qQrhe4ANoksnqw4JBuE1yYyszGnXMsAXwXeDRwUWVxXcUg9pZbdRGbWW5TrnaSlJBYmLTPtQtAB\nvBxFh2i8SqLXORelOuVCuVYETiN1De8XWfysreevUS46kYuBmTVFuQS8H/gGcBWwcWTxXLVR2Uh5\nzMDMRk25JpD2GngtaSmJX1cckjG67063DMxsxIrWwMdIVwWeA3wgsnix2qisGS4GHcL9oSXnolRF\nLpRrInAusCLwjsji7naefzD+XDTHVxOZ2bAUC8sdRpoz8DPgTXUpBNY8jxmY2QIp1wakyWNzSWMD\nD1Yckg3BYwZmNqaUaxHg08CRQAacHVnMrTYqawV3E3UIr7tSci5KrcyFcm0O3Aa8BXhDZHFmnQuB\nPxfNccvAzOajXIsDJwD7k1oFl0VW4/5kGxMeMzCzeZRrG9Iqw/cAh0QWj1ccko2CxwzMbFSUayng\nS8AewKGRxTVjclzpPmACMBuYFBF/GYvj2tjzmEGHcH9oybkojUUulGtH0sJyS5MWlhuTQlCYACwL\nrAzcPIbHfQV/LprjloFZj1Ku5YFTgB2AKZHFj1twmtnFz+eA7VpwfBsjHjMw60HK9V7gDOBa4PjI\n4pmWnEdah9Qi2M5dRO0zmu9OFwOzHqJcq5KKwCakyWO3VByStYA3t+li7g8tdWMuJN0naZakJ4u/\npof7vsnDel0uKdeHgRnAH4HNuq0QdOPnop08ZmBWD30DrZC6VdYeqwMr19rAN4HVgXdGFneO1bGt\ne7ibyKwGJD1JuuLmOWDDsehfV66FgCnA50k7kE2LLF5q9rhWf55nYNa5JjGGA63KtT5pYblFge0j\ni3ubPaZ1N48ZdAj3h5a6MRcR8ZeIWHukhaB/LpRrYeX6NPBr0pVC2/ZKIejGz0U7uWVg1iWUaxPS\nUhL/BCZFFg9VHJJ1EI8ZmHU45VoM+AzwSWAqcKEXluttHjMw6zHK9SbgQuB/gU0ji0crDsk6lMcM\nOoT7Q0vOBSjXeOU6hT/xQ9LVQu/t9ULgz0Vz3DIw6zDK9VbgPODX/IT94tL4XtUxWefzmIFZh1Cu\nZYGvATsDn4wsflBxSFZTXo7CrEsp17tJG87MJS0z7UJgY8rdRB1C0uSI+HnVcdRBL+VCuVYGvgG8\nEfhwZPP/3r2UiwVxLprjYmBWQ8ol4APAqcBlwCaRxfPVRmXdzGMGZjWjXGsAZwPrkpaZvq3ikKzD\neJ6BWQcrWgMHkPYiPhPYI7KYPfS7zMaGi0GHcH9oqRtzoVzrkRaWWwp4a2Tx+2G9rwtzMVrORXNc\nDMwqpFzjgMNIy0l8BTgtsni52qisF7VlzEDShcC7gCcjYuPisRNJTeKnipcdFxE/6vc+jxlY11Ku\nDUkLy70AHBBZ/LHikKxL1HYPZEnbAc8ClzYUgwx4JiJOGeJ9LgbWdZRrUeBYUovgs8B5kcXcaqOy\nbtKSSWeSxkn6iKQfFvu03ifpHkkXSdpqOCeJiJuBpwc6/EiC7WVed6XUyblQri2B24GtgC0ii3Oa\nKQSdnIux5lw0Z8hiIGlV4AfAmsDBEbFBRGwAbEraU/UTks5q4vyHSrpb0gWSlmviOGa1plxLKNc0\n0v9P04BdI4uHKw7LbJ4hu4kkrQf8OyIeG+I120fELxZ4ImkicH1DN9EqlOMFXwAmRMT+/d7jbiLr\neMr1FtKVQncBh0YWT1YcknW5MZ9nEBF/XtABhlMIBnnfvP8hJJ0PXD/Q6yRdDMws7s4CpvddPtbX\nLPR936/l/RX1Trbn42zKJOAQTmQW8HoynqxFfL7fNfeL2x8lmckotG0G8gAtgwl9LQ5JRwCTIuKD\n/d7jlkHB11CXOiEXyrULqSv1J8DRkcWslpynA3LRLs5FqbYzkCVdCWwPrCTpYSADJkvaDAjgIWBK\nO2IxayXlWpG0ntB2pKUkflpxSGbDMuyWgaQzI+Lg4vY0YBJwfkRc3rLg3DKwDlEsJbE7cDrwbeCz\nkcWz1UZlvWrM5xlI+lZE7FPc/nVEvLnhuXWBH0XEa0cb8AKDczGwDqBcE0hrCW1Aag38quKQrMe1\nYp7BqwZ7IiIeAv42kpPZ6Pka6lJdcqFcUq6PAXcD9wKbt7sQ1CUXdeBcNGckYwaLS/qvfu8dP8bx\nmHUE5ZoInAusBOwYWUyvNCCzJi2om2he15Ck3wD7wrxZwwH8NSL+3bLg3E1kNaNcCwEHky6C+Dpw\ncmTxUrVRmc2v1VcTRUQ8MMKYzLqGcr2WtLAcwDaR+f8H6x4LXJuowZJKaxL9StJpxbwBaxP3h5ba\nnQvlWkS5jgNuBa4C3lKXQuDPRcm5aM5IWgbPR8Sbi2UkdgVulPSBiPhdi2Izq5xybU5qDTwFbBlZ\nzKw2IrPWWNCYQePcgv6Xlm4CXAFsEdGarfk8ZmBVUa7Fgc8BHweOAS6JrMYbhps1GPMxg75CUPhF\nv+dmKK1YuhLw6EhOalZnyrU1qTVwL7BJZPF4xSGZtdxw9jM4WtLCwAmSzige+0Lx9CPAhBbGZwX3\nh5ZalQvlWkq5vgFcA3wusti97oXAn4uSc9Gc4YwZvAv4MzAH2LB4bNvi5w7AdS2Iy6ytlOsdpHkD\nvwQ2iiz+XnFIZm01aDGQ9Drg1X13Yf5dySQJeCOpP9VazKsxlsYyF8q1PGm+wNuBAyOLG8bq2O3g\nz0XJuWjOUN1EDwNHAYuQJpj1Hzz7FHBJhCfcWGdSrvcA95A2pN+o0wqB2VgatGUQEc9J2oW0X+su\nwFxgZUkfADYHbo2IU9sTpnmt9kTSfcBawPOkPTD+MuJj5FoV+E9gM2DvyOKXYxtl+/hzUXIumjPk\nAHJEvEAqAi8ALxa3nycVkTUljWTSmtlYmEBaE2tl4OaRvLFYWG4fYAZpD41NO7kQmI2lQecZSFqC\nNNtyWdJfUXOAwyNiB0k3AVcC60TEZ1oWnOcZWD+SniQVgueADYfbMlCutYBzgDVIy0zf0boozao1\n1ktYb0TasSkYYMwgIs4F3iJp2ZEGataESaTxrGEVAuVaSLkOBH4H/BqY5EJg9kqDFoOIuL2h/21u\n8a+/7wPvaEFc1o+voU6KAvCRYRaC9YH/IW0UPjmy+EJkrZktXxV/LkrORXOGM8/g6oj4LoCkvi/+\nHxU/f07qRjKrDeVaGDgCOBY4CTg9sphTbVRm9TbsPZCr4DEDGynl2hi4EPgX8PHI4s8Vh2TWdq3e\nz8CstpRrMeB44CDgOOACLyxnNny+NLRDuD+01D8XyvUm4E7S/JfNIovze6UQ+HNRci6aM2TLQNKr\ngH9HxKCrkkraNiJuGfPIzBZAuZYEvgB8iDQj/upeKQJmY21B+xmsBlxMmtxzRUQ81PDclsCBwMsR\ncWBLgvOYgQ1CuXYAzgNuAw6PLJ6qOCSz2hjNd+cCB5CL5as/UPxbj7Rg3VzSddvntLJV4GJg/SnX\nssA04J3AQZHF9RWHZFY7LSkGVXIxKHndFVCuXYGzmcF0NmGfyOKfVcdUNX8uSs5FyVcTWVdSrpWB\n04CtgH25lrlxjQuB2Vhyy8BqS7kE7EUqBJcDJ0QWz1UblVn9uWVgXUO51gDOIm2w9B+RxW8rDsms\nqw05z0DSqpJWLW5L0jWSVpG0TvFv7faEab1yDXWxzPQBwHTgbmCL/oWgV3IxHM5FyblozoJaBtsC\nn5N0JDAb+Dtps/A/Fs/vTFpf3qxpyrUe6XLRZYG3RRYzKg7JrGcsaJ7B7sC9pKWslwM+AlwQEdsV\nz9/cd7slwXnMoCco1zjgUOCzwFeBUyOLl6uNyqxztWLMYBzwT+AG4GhgxFsMmg1FuV4PXAC8BGwd\nWTxYcUhmPWnQMYNiS8vPAV8n7Sx1HHBYm+KyfrqtP1S5FlGuzwG/BC4l7TcwrELQbblohnNRci6a\nM2jLICLmSnoTcCLwGOnSvp8Bvy+2vQRwU95GTLneQFpm+lHSAPFfKw7JrOcNZzmK44BfAOsC6wNn\nR8QTbYjNYwZdRrmWADLgY6Rux295YTmzsTfmYwaS9iVtGfg0aVG6U4CdpXnniIi4dBiBXQi8C3gy\nIjYuHlsBuBpYB5gJ7BkRs0YSvHUO5dqONDYwHdgksvb8QWFmw7Og/QzmAAGsQRpMPgBYitQ99HLx\n/HBcRLoMtdFU4CcR8RpS99PUYR6rJ3Vqf6hyLa1cZwJXAcdGFns2Wwg6NRet4FyUnIvmDNkyiIhv\nSdoaeIB0Wemqxc+jIuKx4Z4kIm6WNLHfw7sB2xe3LyHtp+yC0EWUayfgXFKx3yiyeLrikMxsEMMZ\nM1gGuJ70l/3s4uG+N0UMc3Gjohhc39BN9HRELF/cFvCPvvsN7/GYQQdSrhVIc1O2Bz4RWdxYcUhm\nPaUVYwbLky77OwL4ErBZv5fMBnYayQkHEhEhyQOJXUC59gBOB75Dag08W3FIZjYMC5p09j/A3hFx\nP2kzm7H0hKTVIuJxSROAJwd6kaSLSQPMALOA6X1rlvf1EfbC/cb+0DrE84r4cq3GH7iaRVmX9Xl/\nZHGrpMk6UWN+vv45qcPvX+H9zSLitBrFU+X9T9Hb3w8fJZnJKCxoOYqDgNeR9pfdGFiaNCP5wYh4\ncUQnemU30TTg7xHxVUlTgeUiYmq/97ibqKCabtxRLDO9L2n3sfOBz0cWL7T0nDXNRRWci5JzURrN\nd+dwxgx2Jg323kHa9nJ5YFPShKGpEfHnYQR2Jan/eCXgCeAE4HvAt4G1GeTSUheDelOudYBzSBcW\n7BdZ3FVxSGZGi4pBceDPAPdFxLUNj00Gjo2IXUYa6LCDczGoJeVaCDiINDv9FOBrkcVLlQZlZvOM\neTGQdB3pssC7SfML+i4NDEAAEXHrqKIdTnAuBvPUpQmsXK8ldQctBOwfWdzf9hhqkos6cC5KzkVp\nzK8mAvYE9iAtK7wmafJY/78AW1YMrD6Ua2HSEhJHAzlwVmQx3EmHZlZzw+0mEmk9mY2BI2M4bxoD\nbhnUg3JtRlpK4h/AxyOLmdVGZGZDadmYQVVcDKqlXIuTNpyZAhwLXDTYwnKS7iPtejcbmBQR3vvC\nrCKj+e5c0NpEVhON19i35Xy53gzcBWwIbBpZXLiAFUYnkLarXBm4uaWxeQ2aeZyLknPRnAWNGViP\nUa7xwEnAXqTNjL4zzGWm+5YqeQ5o2VaoZtYa7iayeZTr7aSF5W4FPhVZ/H3Y75XWIbUItnMXkVm1\nPGZgo6JcywEnA+8ADowsflhxSGbWBI8ZdLFW9Ycq138A9wAvkhaWq30hcN9wybkoORfN8ZhBj1Ku\nVUiri74B+FBk8YuKQzKzCrmbqMcUC8t9kLSMxCXAiZHF89VGZWZjqRUzkK2LKNdawNmkxQF3jSxu\nrzgkM6sJjxl0iGb6Q5VrIeWaQtqT4jZgy04uBO4bLjkXJeeiOW4ZdDnlejVwHrAksENkcU/FIZlZ\nDXnMoEsp1zjSdqVTSVuWfsMLy5n1Bo8ZGADKtRFwIWk28FaRxR8rDsnMas5jBh1iOP2hyrWocmXA\nTaQ9B97WjYXAfcMl56LkXDTHLYMuoVyTSK2BmcDmkcUj1UZkZp3EYwYdTrmWBD4PfJg0RnDlMBeW\nM7Mu5TGDHqNck0lXCt1BWkriqWojMrNO5WLQIRr3d1WuZYBpwK7AQZHFdVXG1m7e67bkXJSci+Z4\nALnDKNc7SQvLjSO1BgYtBJLukzRL0pPFEtNmZgPymEGHUK6VgNOANwOfiCx+tsD3SLNIu48BPBwR\na7cwRDOrCY8ZdKFiYbk9SYXgKmCTyOK5Yb7du4+Z2bC4ZVBjyrU6cBbwGq7jjLgzzhrR+7t09zH3\nDZeci5JzUfLmNl1CuaRc+wPTgRnA5vyOe0d6nIj4S0Ss3U2FwMxawy2DmlGudUmXiy4H7B9Z3F1x\nSGbWYdwy6GDKNU65DgduB24krSnkQmBmbeFiUAPKtQGpb38PYOvIYlpk8fJ8r/G6K/M4FyXnouRc\nNMdXE1VIuRYBjiEtI3EC8M3IYm61UZlZL/KYQUWUawvSwnKPA1Mi8yCvmY0NzzPoAMq1OJAB+wNH\nA5d5YTkzq5rHDNpIubYF7gZeTZo8dulwC4H7Q0vORcm5KDkXzXHLoA2Ua2nS1pO7A4dEFtdWHJKZ\n2Xw8ZtBiyrUjcC5p97GjIot/VBySmXU5jxnUiHKtAJwM7EAaIP5xxSGZmQ2q8jEDSTMlzZB0l6Tb\nqo5nLCjX+0jLTD8LbDwWhcD9oSXnouRclJyL5tShZRDA5IjO7z5RrtWAM4CNgT0ji1sqDsnMbFgq\nHzOQ9BCwZUT8fYDnOmLMoFhm+sPA10hzB/LI4oVqozKzXtWpYwYB/FTSHOCciDiv6oBGQrnWBs4B\nJgC7RBa/qzgkM7MRq0PLYEJEPCZpZeAnwKERcXPxXACXADOLl88Cps/bC7joI6zivnItxK84mdXY\nj/WYBkzjRLZp2fka+kPr8PtXeb9/TqqOp+L7m0XEaTWKp8r7n6Im3w/tvl/c/ijJTCAbacug8mLQ\nSFIGPBsRJxf3a9lNpFyvAc4ntaz2jyzua/k5vXHHPM5FybkoORel0Xx3VloMJC0JjIuIZySNJy3d\nnEfEjcXztSoGyrUwcBTwaeALwBmRxZxqozIzm18njhmsCvy3pL5YLu8rBHWjXJsCF5C6qiZFFg9V\nHJKZ2ZjH7j71AAAJVUlEQVSpVTdRf3VoGSjXYsBngSnAVOCiKhaWcxO45FyUnIuSc1HqxJZBrSnX\nVqRLRR8ENossHq04JDOzlnDLYKDz5hoPfBHYGzgc+LaXmTazTuGWwRhQrreSNqT/FbBRZPG3ikMy\nM2s5F4OCci1HmkG8E/DJyOIHFYc0H/eHlpyLknNRci6aU/lCdXWgXLuRFpabQ2oN1KoQmJm1Wk+P\nGSjXysDpwCTggMj8V4WZdb7RfHf2ZMtAuaRcHwR+DzxC2oLy59VGZWZWnZ4rBsq1JnAdcBywW2Tx\n6cji+YrDWiCv1V5yLkrORcm5aE7PDCAXy0x/HDiJtOfA7pHF7GqjMjOrh54YM1CuV5EuF10K2C+y\nuKfp4MzMasrzDPpRrnGkSWPHA18GTvPCcmZmr9S1YwbKtSFp4thuwFaRxcmdXAjcH1pyLkrORcm5\naE7XtQyUa1HSgnKHkhaYOy+ymFttVGZm9dZVYwbKNYm0zPTDwIGRxcMtC87MrKZ6dsxAuZYAcmBf\n4EjgCi8sZ2Y2fB0/ZqBc2wMzgLWBjSOLy7uxELg/tORclJyLknPRnI5tGSjXMsBXSQPEB0UW36s4\nJDOzjtWRYwbKtQtwDmnP5KMji1ltD87MrKa6fsxAuVYETgW2JU0e+2nFIZmZdYWOGDMoFpbbk7TM\n9D9IYwM9VQjcH1pyLkrORcm5aE7tWwbKNQE4C3gdaT2hX1UckplZ16n9mAEn8hRwLvDFyOKFqmMy\nM6u7bh0z2DGymF51EGZm3az2YwYuBIn7Q0vORcm5KDkXzal9MTAzs9ar/ZhBK/dANjPrRt4D2czM\nRsXFoEO4P7TkXJSci5Jz0RwXAzMz85iBmVm38ZiBmZmNiotBh3B/aMm5KDkXJeeiOS4GZmbmMQMz\ns27jMQMzMxuVSouBpJ0l3S/pfyUdW2Usdef+0JJzUXIuSs5FcyorBpLGAWcAOwOvB/aWtEFV8XSA\nzaoOoEaci5JzUXIumlBly+CNwB8jYmZEvARcBfxHhfHU3XJVB1AjzkXJuSg5F02oshisATzccP+R\n4jEzM2uzKotBfS9jqqeJVQdQIxOrDqBGJlYdQI1MrDqATlblTmf/B6zVcH8tUutgPpJcNAqS9q06\nhrpwLkrORcm5GL3K5hlIWhh4AHgb8ChwG7B3RNxXSUBmZj2sspZBRLws6RDgx8A44AIXAjOzatR6\nBrKZmbVHbWcge0JaSdJMSTMk3SXptqrjaSdJF0p6QtLvGx5bQdJPJD0o6UZJPXFJ4SC5OFHSI8Vn\n4y5JO1cZY7tIWkvSTZL+IOkeSYcVj/fcZ2OIXIzos1HLlkExIe0B4O2kgebb6eHxBEkPAW+IiH9U\nHUu7SdoOeBa4NCI2Lh6bBvwtIqYVfygsHxFTq4yzHQbJRQY8ExGnVBpcm0laDVgtIqZLWgq4E3gP\n8DF67LMxRC72ZASfjbq2DDwh7ZV6csG+iLgZeLrfw7sBlxS3LyF98LveILmAHvxsRMTjETG9uP0s\ncB9pnlLPfTaGyAWM4LNR12LgCWnzC+Cnku6Q9PGqg6mBVSPiieL2E8CqVQZTA4dKulvSBb3QLdKf\npInA5sBv6fHPRkMuflM8NOzPRl2LQf36rqq1TURsDuwCHFx0FxgQqZ+zlz8vZwPrktbleQw4udpw\n2qvoFrkGODwinml8rtc+G0UuvkPKxbOM8LNR12IwrAlpvSIiHit+PgX8N6kbrZc9UfSTImkC8GTF\n8VQmIp6MAnA+PfTZkLQIqRBcFhHfLR7uyc9GQy6+1ZeLkX426loM7gDWlzRR0qLAXsB1FcdUCUlL\nSlq6uD0e2BH4/dDv6nrXAX0zTfcFvjvEa7ta8YXX5730yGdDkoALgHsj4rSGp3ruszFYLkb62ajl\n1UQAknYBTqOckPblikOqhKR1Sa0BSJMEL++lXEi6EtgeWInUB3wC8D3g28DawExgz4iYVVWM7TJA\nLjJgMqkbIICHgCkNfeZdS9K2wC+BGZRdQceRVjLoqc/GILk4HtibEXw2alsMzMysferaTWRmZm3k\nYmBmZi4GZmbmYmBmZrgYmJkZLgZmZoaLgfUoSbmkJaqOw6wuqtwD2WxMFRP0Vm94aHfgGeDGhsf+\nDmwBrAJsVCyH3WdFYEJErDzI8VcDxkfEnwZ47tXAZyPio8X9rwP/FRG/XUDMe5CWJz9ugOfWA57r\nhUlkVj0XA+smywFrFrcFLAss0vAYpD23dwTeHRFPAzsASFoRuB54/0AHLvbsvhjYr9/jXwG2ARYH\n1pJ0c/HURODtkp4B7ifNGN5pgEMvDywp6e3AHRHxyYbn/g1cIuldETFnQb+8WTM8A9m6iqT9gQNI\nm8CMJ03Ffx5YkrQvxlrAV4GzgBMi4gFJWwOXA1dExGcGOe4+wDoRcdIgz68CvI+0lvwKwBLAbxtb\nEcUaMotFxAsNj+1OahkcP8hxjwceiYhLh58Fs5Fzy8C6zVLATaT13Bs39tgEWCYijpH0DuBfwHhJ\nl5O+uHcFDpP0fdISwP27gj4AHDrQCSV9EXgXMIvU/bQw8ChwhKTfRETf+3YszrFbv7/0h9qA5Erg\nTMDFwFrKA8jWjeYU/14u/s0B5gJIWp3U3XMVqcvnjIh4X0T8ISKmkNZ833+AY64XEQ8Ncc5PRcQO\nwJeBU4vb821EFBE/Ji0Ydt5wf5HinOsO9/Vmo+WWgXUbAe8Gtur3+ArAtcBlpB2xXia1IL6Uem+Y\nQNoABODTgxz3lQ9K/0naanE3Sf+k2FlL0odI3VSrSlouIj5cvOVw4PuSXh8R947qNzRrARcD6zZB\n2uTjVub/At8cWIy0N8bBABFxTfFaJN0QEbsMcdy5A54sdQEdWhxjbVJ3ztPAoRHxig2Ziu6hoc4z\nH0njSC0bs5ZyMbBuI9Jf5MsXt6P4uTQwOyL+VrQERup3kraMiDteccK06dB+pPXj9wVeBC6TdDVw\n1QLW01+DNNg9mC2B20cTsNlIuBhY15B0FvBWYDbpcs9GS5EGjDcF7ilef1PD8+Mb7j8UEfv1e/85\nwIGkK5Uaz3kssCdwCbB9RLxUPL4T8DHgv4tB5OMa3nMUcAzpKqcXSd1ag5lSnNuspXxpqdkwSfom\naY/ZWxoeWywiXmzR+bYH9oqIg1pxfLNGLgZmZuZLS83MzMXAzMxwMTAzM1wMzMwMFwMzM8PFwMzM\ncDEwMzPg/wHahXLZNvMNCQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x798ac50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt = runplt()\n",
    "plt.plot(X, y, 'k.')\n",
    "X2 = [[0], [10], [14], [25]]\n",
    "model = LinearRegression()\n",
    "model.fit(X, y)\n",
    "y2 = model.predict(X2)\n",
    "plt.plot(X, y, 'k.')\n",
    "plt.plot(X2, y2, 'g-')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一元线性回归拟合模型的参数估计常用方法是普通最小二乘法（ordinary least squares ）或线性最小二乘法（linear least squares）。首先，我们定义出拟合成本函数，然后对参数进行数理统计。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###带成本函数的模型拟合评估"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下图是由若干参数生成的回归直线。如何判断哪一条直线才是最佳拟合呢？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEaCAYAAADzDTuZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm4HFW19/HvjwQhA4QEAwkQCJMoioKCF4RIhSEQhohi\nQBExIFy4DImgTA50WlAGBfGCIAgyqIAoDuAjCHhTGEAREQwgwysGZAxjIGEKSdb7x66T6nTO2N3V\nVd29Ps9znnRXd1etsyl6nz2tLTPDOedcZ1sp7wCcc87lzysD55xzXhk455zzysA55xxeGTjnnMMr\nA+ecc3hl4NxyJDXt/wlJZUlDavjc4CzicZ3NbypXCJJWA2YDvS18eT9wR/J4K+Debh4/YmZHSFoV\nuNbMpiTnPxI4DNjJzF6puO6BwCAzu0LSUOAuSacCS5K33G5m85L33mNmH+nld9gQWKfi0L7AAuDm\nimMvmdnDkg4A1gI+IOmsitfXBMaa2WhJGwEfBYYB84ExwPVASdL3zez+imufBtwATAb2Sq7bZSxw\nopn9tqfYnfPKwBXFIOAVM5soaSIwycxOlnS3mW1T/WZJJ5jZWcnj883s6Kq3fBr4Y/L67sB7gc8C\nV0r6nJm9JukCYKfwFq0DjATuB24HRgBXA7sl5xgEbCrpzz3E/w/gImC9rhCTc6xccawr9knAUcBe\nScU0MTm+JuELfWry1rmECuUVYDVguJk9KWk7YJakDxIqiZHA9oSKZFXgcDO7p+J6X+whZueW8crA\nFdEqwDvVByVtCswgtB7+UfHSKEnrmdlTyfsEHAOcKOlrwLrA0Wa2VFIJuEHSJWZ2ZNIKeBqYQ2hh\nLCJ8GU8GjutqFSTn+JOZ7dVb4JI+DBwKLCT8RW+EimgocLWZ/VzSd4ApwA8lnWJmj0j6GPAz4Coz\ni5PTnQzsmsT1FPCCpFeBR5LzHw38BHgIeCb5dxPgB5IqWwZrAl/rLW7nvDJwRbQ6y3dzdFmX8Bfy\nU8B7JJ0PbAu8DFwlaXXgbNK/xocTviCfBbYgVCCfB7paCgD7ELqEvgNcAHwOOJ1QQSySJAs5W0YD\n20q6u4eYtzezRck1ZwF/IbQOunyQ0FLAzI6XtCvwGjBM0s+AIYTunemSfkeo9M4g/D+6cvLvGcBJ\nwDwz+62k6cBvklbOv4G7CK2Eo8zsHkknAjeZWWXF6Vy3vDJwRfRewl/qyyRdPVsA/wR2Jnxpzwcu\nBQ4HZgKzzOwmSd8DLiN0O92R9Kc/mpxqS+BtM7s3OecrwCXApoRupRsJf11vAHwFOBbYN+l2eXd3\nwSZjCYsqDi0hHXOAUCksSf4l6ZK6HDgIOAQ438y6up8OlxQBXwQWE77gXwQeA3YEjgO+nwwiD00q\ngvWAzQkV20PAnpJeB8YDU5NWwiNmdkR38TsHXhm4gpG0FvBJ4LvJoXeSvvQtCV/caxAqgEMIXShG\nGFQF+LCkCYQvzFMqTjsSeLWbyx0E/JzwJX0hYUB3SvLTZd+B/grA3oQWS6VRwHXJ458SvuQXE1oR\n3w49W4wltGIATgAOSH6PDYEHgQVm9lVJ/yS0gLoqkK8TKqoZhC6u48zsqaRL7HeV4wfO9cQrA1cU\nRpjq/BNghpm9kRw/H/gd4a/jywldRfMIX9hLzWxSdydLvly7ZiZ9iLRlUOnLhAHkwWY2T9L7zGxC\nxTkuA4ZI+hUwroeYyyzfHWSEL/07qo5vRRgLAdgfOBLAzK5L3o+kG81scsX1HybMsNowuf6JyUsX\nAPcBmyXnOCIZ+3g9ed/hFd1QO0haH/hvM7utm9/BOcArA1ccApYCk81saddBM7sKuAog6T55F+EL\n/Feks256PGfymWfM7O3qF83s2aTS6JWZfaqXC2wGPFf1ewwjtEZEqBxEGOtYlJzvhX5ed4Gkkwnd\nV6OBSck6iBMJYxLHSjoqGdNYiTAm8RahxXFUV4tA0pf7vJjreJlXBpLGAVcSmuAGXGxm/ytpJmHW\nxQvJW082s5uyjscV1mJCF9A9vXxRrkoYC/geYQrmof0474mEaZxdFrHiWoZlzyXNrji+CVCqPqGk\nMuGv+7cJs4SOTo53TVVdRJjqWWk4YbD4g2a2f8W5ZlW8Z1jF87mE8YsDgDuTnxnAe4A7zexzki4E\nfinpCUKXmYD/JUxVrZxRNA7oaeDbOQCU9eY2ksYAY8zsPknDgXsIA137EfpAz8k0AOdalKRtgIfN\nbEHFsY+a2V8rnq9vZv9JHnfNfHJuwDJvGZjZcyTNaDNbKOkhQr8vLN+n6pyrYGYr/DVfWREkz/9T\n8dgrAlezpuYmkjSeMJD2l+TQMZL+IelSSWs0MxbnnHOpZiblGg78kjBTZCFhKt+GhCmDzxKmyjnn\nnMtB5mMGAJJWJkwPvNHMzu3m9fHADWa2RdVxb/Y651wNzGxA3fDNmE0kwiKhf1ZWBJLGmlnXAptP\nEhKErWCgv1C7kjTTzGbmHUcReFmk2rUsJPYmfG8cYMat/ftM65SFyhLw38CVVrI3G37+Gv6QbkY3\n0fbAgcBESfcmP5OBMyXNkfQPwjL7Y5sQSysbn3cABTI+7wAKZHzeATSaxDTgR8Ce/a0IEuMzCSgD\nVjIjJBAckXcsXZoxm+h2uq90bsz62s651iLxFULG2ciMh/OOJ0tWsm/nHUMl3+msdVyedwAFcnne\nARTI5XkH0AgSkjiLkHNq+xorgssbG1XjqKxN846hL00ZQK5VsobGxwyca2MSg4GLCZlX9zTjpZxD\nahiVNZSQv+rzwEesZE835bo1fHd6y6BFJDl2HF4WlVq9LCSGEKacrwvsUk9FUNCy+Drhd9uiWRVB\nrTxRnXMuFxIjCOnHnwH2M2NRHx9pRadYyRbnHUR/eDeRc67pJMYANxFSdM8wY2kfH3ED4N1EzrnC\nk9gIuJ2Qhnx6u1QEKusbKutzecdRK68MWkRB+0Nz4WWRarWykPggoTVwthnfNFshnXgd5869LK4E\nfpNzDDXzMQPnXFNITCDs6na0GdfmHU+jWcmeyDuGeviYgXMuc7WklyiqJJXEwcB9VrK/5x1Pd3zM\nwDlXOHWklygclbUBcDNhD+sVtlJtZV4ZtIgC9IcWhpdFquhlkaSXKBPSS2S69WaTymJlQmWwrZXs\nwSZcr2l8zMA513ASAs4A9iakl3gq55Aawkr2L+A7eceRBR8zcM41VLukl1BZKwPrWcnm5h3LQPmY\ngXMuVxXpJdahzvQSBRABx+cdRLN4ZdAiit433ExeFqkilUWSXuIm4A1gihkLm3v9xpaFlewW4KhG\nnrPIvDJwztUtSS9xGzAHOLAV8wyprEHVx5JNaDqCjxk45+qSpJe4mbAC99RGripuBpW1GvBtYFUr\n2WF5x9MIPmbgnGuqLNNLNENSEfwDGAackHM4ufLKoEUUqW84b14WqTzLIkkvcStwrBkX5hVHGs/A\ny8JKtgDY00p2iJXslcZH1Tp8nYFzbsDaKb2EleyhvGMoAh8zcM4NSJJe4gxg76xXFTeSylodmAGc\n1u4Dw7V8d3rLwDnXb0l6iWMI6SVq2bQ+T28ArxK+997JOZbC8ZZBi5AUmVmcdxxF4GWRalZZVKWX\nmFTE9BJ+X6R8NpFzruGS9BKXAjsCE4pYEVRTWSuprO3yjqOVeMvAOdejJL3E1cCqwKebvaq4Firr\nPcAlydOdWmVD+kbyloFzrmHyTi9Rh9OAXwATO7EiqJVXBi3C59anvCxSWZVFK6aXqCiL/a1k51nJ\nluQZT6vxysA5t5wkvcTtwK+A6WYszTmkHnV6PqFG8jED59wySXqJG4HTirCquC8q6w/A8VayOXnH\nUiS+zsA5V7MkvcQvgWPMuDbvePppGvBc3kG0A28ZtAifQ53yskg1qizaIb2E3xcpn03knBuwJL3E\nxYQtKgtZEaisNVTWeSprnbxjaVdeGbQI/4sn5WWRqrcskvQSM4GJRc0zpLJ2BR4gdGv3OL3V74v6\n+JiBcx2oKr3EDgVfVfw6cKCV/Ms+Sz5m0CK8PzTlZZGqpSyS9BIXA5sTuoZaedP6Zfy+SPlsIudc\nr6rSS+xStFXFKms9YJCV7Im8Y+k0mY8ZSBonaZakByU9IGl6cnyUpFskPSrpZklrZB1LK/O/eFJe\nFqmBlEWLpJfYB5hQywf9vqhP5t1EksYAY8zsPknDgXsI/8EPBl40s7MknQiMNLOTqj7r3UTONUCS\nXuImwn7FM4q8qtjVr5BTS83sOTO7L3m8EHgIWBeYAlyRvO0KQgXheuD5eFJeFqn+lEVR00uorEEq\na5WGnc/vi7o0dWqppPHAVsBdwNpmNi95aR6wdjNjca4TJOklZgNnm/FNMwoxY0RlfQC4Ezgk71hc\n0LQB5KSL6DpghpktkNIWjJmZpG5vUkmXA48nT+cD93X1DXb9JdAJz80sLlI8/rw4z7us+PpRx8A+\nZdj1CDOuLUy8M3kVuJk7uZJbeZhST/EP7HnXsbx/vzyeJ4+nJcXwODVoytRSSSsDvwNuNLNzk2MP\nA5GZPSdpLDDLzN5b9TkfM3CuBkVOL6GyBKxtJfOcQhkp5JiBQhPgUuCfXRVB4nrgC8njLwC/yTqW\nVub9oSkvi1R3ZVH09BJWMsuiIvD7oj7NGDPYHjgQmCjp3uRnd8Lqx10lPQrslDx3ztWhaOklVNbm\nKusbecfh+uYrkJ1rA1XpJSYVJb2EyhoBTLCS/S7vWDpJLd+dXhk41+KS9BIXAe+njdJLuNoVcszA\nNYb3h6a8LFLS2EmEDWnWJaSXyK0iUFlrqqyJuV3f74u6eGXgXIsK6SV+eBY5p5dQWVJZ+xHSTO+U\nRwyuft5N5FwLKlJ6iWQV8S+AM6xkd+YVh0v5mIFzHSBJL3EzIY3LaUVZVeyKw8cM2pj3h6Y6uSyq\n0kucCtqx6TGUNaTZ1+yPTr4vGsErA+dahMQE4BbgWDMuzCWGsoYCf1dZw/O4vsuOdxM51wKKlF5C\nZQ23khVxLwSX8DED59pQkl7idMKModxXFbvi8zGDNub9oalOKou+0ktkWRYqayuVdV1RxwiqddJ9\nkQWvDJwrIAlJnEnI97+DGQ839fplnQz8gZBQ8q1mXtvVJ45VU29K0/YzcPWpzNne6dq9LKrSS0zo\nbVVxhmVxN/DBVkoz3e73RX/EsY4HDqvls14ZOJcz6QN7wLjpsNqq8MY7cOhw2OdVQnqJXAZqrWQN\nGaSW9BAwFlgEbGNmTzTivK5H/wIOgIGPLfkAcouo3MGp07VTWYSKYLvvw482SY9+ZSHM/pzZXdf3\n/fn6y0Jl7QL8zUo2v57zdHtuaT4wInn6pJmt3+hrVFyrbe6LvsSxpgKjosgu6u51H0B2ruWMm758\nRQDw3eGw5v80MYgdgA0yOvei5N/XgQkZXaMT3QONnWLsLQPnciTtF8O13awi3u82s2ujZsfTaJI2\nIKyYnuBdRP0Xx1oZ2A3YF9ggimxACQC9ZeBcy7GVuz++8M0srqay3p3sQdwUZvaEma3vFcGArQoc\nC9xHuj1wpnwAuUV0Un9oX9qlLEJ6iSM2h2Oeg/PGpK8c8gT857z+naN/ZZFUAAcC3yXshvbXmoIu\nsFa9L+JYnwLiKLKXu45FkS0Adm5mHF4ZOJeDNL3EzlNhxrvgo7+FoYND1/ojQ81e+32DL3kIcAyw\nh5Xsngaf29VnM+B+4OW+3pglHzNwrsm6Sy8h6XlgNKE2eH+ju1VU1qrAEivZO408r+ufONZoYB/g\nySiym7K+no8ZOFdwvaSX2AZ4kgwqAgAr2VteEeQjmQb6L2BXIJOxoEbwyqBFeN6VVCuWRV/pJWod\naO2uLFTWVJV1TF0Bt6Ai3BdxrDW6OXwjMCaKbL8ostuaHVN/+ZiBcxlL0ktcDGxOH+klGuRu/A+9\npku6gm6PY703itL+9yhqjXTfPmbgXIYkhgBXE6YK7mvG6zmH5BogjrU58O8osreqjg+KIluSU1jL\n1PLd6S0D5zIiMYKQ9fNpYD+zZatxG3eNsv4LGOQb0TdHHOso4ChgdcKisAcrXy9CRVArb0q2iCL0\nhxZFK5SFxBjgNmAOcGCjKwKVNUxlncNj3AiMauS5W1WT7ou5wMHA+lFkD/b15lbiLQPnGkxiI+Bm\n4ArgNDOy6ItdHViNWzjYrrTfZXD+jhbH2gN4XxTZ2ZXHo8gavf6jMHzMwLkGkvggYfbIaXltWu/q\nF8daDxgeRdbUTYUaxfdAdi5HIb0E1wFHm3FtQ89d1mgr2QuNPGcni2MNBnYhJILbEvho5QygVueL\nztpYK/STN0sRyyJJL3EdcEAGFcH7gF90l2CuiGWRlwGWxUrAccDDwH7tVBHUyscMnKtTRXqJPbvb\ntL5eVrKHVNYuVvIvrFok/f/3RZE903UsimwRMCm/qIrHWwYtohWzMWalSGXRS3qJhrKSLe72eIHK\nIm+9lMV7gHc3MZSW5GMGztVAQsAZhHTQk8x4qu5zhm6gQ4FdrWT71Xu+ThLHGgVMAV6NIvt13vHk\nzccM2pj3DafyLoskvcSlwI6E9BKNqAgGATcBhwGn9j8Wvy/iWLsBc2+7jUPAV3jXyisD5wYgSS/x\nS2AdYOdG5Rmyki0BvgV8zEp2fyPO2Y56SAT3J2DdmTM5JYrs5mbH1C6a0k0k6cfAnsDzZrZFcmwm\noUncNV3uZLPl83x7N5Erkqr0EtOySC/hehbHGgY8ALwnijwdd2+K3E10GbB71TEDzjGzrZKfzDd8\ncK5WjUwvobIGqazDVZbP5utBHGvTONbwymNRZK8Dm3hFkI0+KwNJgyQdJOn3kh5Kfh6QdJmkbftz\nETObDbzS3ekHGnCn8r7hVLPLIkkvcTthHcF0M5bWecqlwLrAavXH1l73RRzr4DjWHELXz+bVr/eW\nCK7dyqLZev3LRNLahPwqfwKOMrO5yfFBwNbA4ZIOMrMja7z+MZIOAv4GfNnM5td4HucyIfEh4Pc0\nML1Esl7glEacqw09CRwJ/LmVM4C2ol7HDCRtBLxpZs/28p4dzfrevUfSeOCGijGDtUjHC04FxprZ\nF6s+42MGLjeNSi+hsja1kv2/xkXW+uJYOwITo8hm5h1LO6rpu9PMmvIDjAfuH8hrhHGFywmLemYC\nXwKiitcjf+7Ps3gOtjfc/Aoc++WazzeKPfgkv2EmTzGTkUX6/fJ+PmsWa02cyMFFiafVnyePLyf9\nvrSu9/X3p2mLzrppGYy1pMUh6VhgGzM7oOozZt4yAEJ/qPlqUyD7sqhILzHF6lhVrLIuBN4FfMVK\n1t2YWd2Kel/EsVYCdiIkgpsAbBlF3a+ibpSilkUeCrvTmaSrCQt03i3pSaAERJK2JPz1Pxc4vBmx\nONebJL3E0YT0EvWmLz6mpzQSHWIGYeD97KwrAle/frcMJP3AzI5KHp8FbANcYmY/yyw4bxm4JknS\nS5wJ7EWD0kt0ijjWzsDcKLJ/5x2LCxq+zkDSTyuefrjrgZmdAByCz4hwbaAivcTHqTG9hMo6W2Xt\n0fDgWsPGwOi8g3D16aubaOOeXjCzuZJebHA8rgfeH5pqZFkk6SWuAVYhpJeoNbfNhdD81kSz7os4\n1ghCUr6lUWRXVb4WRXZx1tfvD/9/pD4DGTNYVdIvqj47rMHxONc0Veklplodq4qtZP9qWGAFE8f6\nGCGJXgxckm80LjO9TTUC/lzx+C/AZsB7k5/NgCEDnb40kB/ALP2ZucJ7YGYvx/1z/rkePwc2Buw+\nsPPeYVC535+byUrM5GhmsmmRf79aPzdrFmtUf27JYOytNTm9SHH653r/HI2eWirpz2a2XfXjZvEB\nZJcFiY2BPxBW158W/n/qx+fK2pSQZwtgWru1BpJ9gR8FPhRFtiDveFztsk5UNzTJSXSnpHOTdQOu\nSTzvSqqeskjSS/wJONuMU/tbESSGE8YXPl6UiqDWsohjbRjHGll5LJn+uUmrVgT+/0h9BjJm8IaZ\nbZekkdgLuFnSZ8zs7xnF5lxD1Ztewkp2L3BvwwNrojjW/sAJwDjgM8D/Vb4eRVZvEj7XovrqJqpc\nW7BcN5GkDwJXAR82s0zyuns3kWsUib0J00cPMOPWPt9f1qrAaCvZk5kH10RxrAnAIOB2XwjWvhq+\nArmrIkjcVvXaHEkXEDaafmYgF3WumSrSS+xp/U8vsQ+wBfC1rOLKUhxra2D/KLLjK49Hkc3OKSRX\ncP3Zz+ArkgYDp0g6PznWtUfrU8DYDONzCe8PTQ2kLCSOJyTumjiAigDg51aywlcEkqI4Vnd/AT4G\nXN3sePLk/4/Upz9jBnsC/waWAO9Pju2Q/DuRME/buUKpSi+xg/WxqlhlDUr2IQaW7TlQWEkFMOGk\nkzgGuDiOtWUU2Rtdr0eRvUL3G0o5160eWwaS3itpr66nVO1KJknARwmJqFzGzFdWLtNXWQwkvYTK\nGqmyfkzoRmoZUWQGHLXbbvwdmFJZEXQq/3+kPr21DJ4kLLFfmZBZtPovpS8BV5j5fqSuOAaSXkJl\nrQ38Hfg1YYOlQopj7QC8FEX2UOXxKLL9cwrJtaEeWwZm9jowGRiR/LsHMFrSZ4CtgDXNipGTpBN4\nf2iQ7MG9UNLzkjZY/jVGENImvE7Yi6DXPENWsnnAJCvZ0VYq9Nz6DYG1u3vB74uUl0V9+ppN9Jak\npcBbhJbBUuCN5HPrSVrJzOclu6YaS8iJNQyYDawPIDGGUBHMBmZYPzett5I9mFGcAxLHWo3wB9ew\nKLIfV74WRfaTfKJynaS3MYMhkn5LGISKCYtTXjKz64G7gTspcNO63Xh/6DJda1peJ+yg1ZVe4g7C\ngrLp3VUEKmu0yjq++ngRxLE+QEiWNw1YOJDP+n2R8rKoT29TSz8AfI90vGC5MYOki+jjkkZkF55z\nK9iGMJ71fjN7oiK9xHf7SC+xEHhbZQ0kBUvDVaeASDwErB9FNjmKbMAro51rhD53OpM0C/g+YWrp\ncWY2UdKs5N8TgcfM7JeZBOcrkJfxXO0gfWAPGDcdXhsDqz8HW/4RTj+eGtNLNFsyHfSfQBRFNq8R\n5/T7IuVlkcoqUd3Pzew3ZnYDcH9y7Kbk3xh4bSAXdK4WoSLY7vtw427wrQ+FfzkdSudWVwQqa7DK\n2ianUAGIY42LY61VeSyZDvr+RlUEzjVSv/dAzoO3DFwXafJNSQVQZY+bzH4/edn7ytqCsMbgeWDv\nZi8ei2NNIaSw2AQ4OIrMF2W6pmt4biLnimO1Vbs/PnxI1yOVJeA7wMXApTmtIn4W+DoQR5GvwXGt\nwyuDFuH9oQveSh/HQJQ8Xvhm11Ermamsyc2oBOJYmwNHR5EdWXk8imwg+Y/q5vdFysuiPr2OGUja\nWNI6fbxnh95ed65eIb3EIYPgxLeWf+XQx+A/51UeyaIi6CER3BPAlY2+lnN56Ws/gzHA5YSFPFeZ\n2dyK17YGjgAWm9kRmQTnYwYdb/n0EtHFMPSw0DW04E32vXN9tnhtTyvZ41lcO461HTAV+ATwX1Fk\nL2ZxHecaLYv9DJ5LktV9BjhP0kaEhHVLCTldLjIzT1TnMpGkl7iesCBrqlm8CPjVstfLWsdKluVe\nGocB/yHsbfBShtdxLnc+m6hFdFp/aG/pJRpdFnGsjwLvRJG13JaWnXZf9MbLIpXVOgPnmqo6vQQz\nNUplna9yt6t3G2EcMCajczvXErxl4AolSS/xe+A0My5UWZ8AfkjYb/sbVqotb38caxiwOzAmiuwH\nDQvYuQLydQaupUlMILQGKtNLvAHsYyW7q9bzxrE2AOYAfwU8A6hz3ehrNtHaAGY2L9nZ7JfA/wBd\nC33MzP6TWXDeMlim3ftDJaYAlwAHmHFr7+/tuSySRHDzk9QPXccEjIwie7mBIRdCu98XA+Flkcqi\nZbAD8A1JxxFSB79E+MvtX8nruxPyyztXM4lpwOm8/9rDmLr//StuqjcgfwAOIL1Hu3ICtV1F4Fwj\n9aeb6LOEVNZrAAcBl5rZwQCSZmcYm6vQrn/xSBwPHAVMZOr++xJqgl7z+ZhZHMdaB1AU2dNVL28b\nRZ2z4VK73he18LKoT1+VwSDgVeBG4CuEVZfO1U1CwJnAXsAOYdN6+1Zfn4tj7QrMBDYnVCJXVb7e\nSRWBc43U205nKwHfAL4LjAZOBqY3KS5XpZ32d5UYjBZfBks/DkwIFUG/zbvoIq4nzAq6qs93t7l2\nui/q5WVRnx4rg2Rv4/8CniJkYvwZMBm4X9KsZNObxU2J0rUNiSEMfeFWxt35KY4df6FZ9yt741gb\nxLGuqD4eRTbnmmu4K4rs7eyjda5z9JWO4g1JrwD3Egbl/gRcaOabczRbO/SHSoxgyIu3MT7ejL0P\nO4Ihr14JYbZP5eyfxDOE2UUraIeyaBQvi5SXRX36mlr6BeCrhAHkI4BzCLmJupiZ9Zm5UdKPgT2B\n581si+TYKODnwAbA48B+Zja/6nM+tbRNLEsvoSW385Wxp9pZz8+LY20N7Ad8CtglirJJOOdcp8ki\nHcUSwuyOdQmDyYcCwwndQ4uT1/vjMsI01EonAbeY2XuAPybPXQ9auT90ufQSNugYO+v5rpblNMKU\n5akMYHJCK5dFo3lZpLws6tNXN9FPJX0MeIQwrXTt5N8vm9mz/b2Imc2WNL7q8BRgx+TxFYQdS7xC\naCMqa2se2G//wYN/ctD73veXK+bM+fipla9HkR2dV2zOueX1mZtI0urADYS/7Bclh7s+ZNbP5EZJ\nZXBDRTfRK2Y2Mnks4OWu5xWf8W6iFqb17/gEz2x9+S4Tf3HB1772+b9HkV2Xd0zOdYJavjv7GjMY\nSdjNqQR8Htiy6i2LzKybTcq7Pdd4eqgMkucvm9moqs94ZdAi4lhDgEnAxlFk5wwkvYRzrrGySEfx\nf8BnzexhwmY2jTRP0phkA52xwPPdvUnS5YQBZoD5wH1dswa6+gg74Xllf2gR4ql8PmsW9y81Hpt1\nB/NeeIobJk5M0ktwyDfgssVdDclGXa+6TPL+/XN+vqWZnVugePJ8/iU6+/thGsHj1KCvlsGRwHuB\nLwFbAKsRViQ/ajawed7dtAzOAl4yszMlnQSsYWYnVX3GWwaJoiThimONAF6PIlsMoLIEHLT6YL7z\n2mLOYqZMB6H/AAASU0lEQVSJsDJ4dzMeziKGopRFEXhZpLwsUg2fTWRmFxByy58PfIQwbnAE8EdJ\n1ypsg9mfwK4G7gQ2k/SkpIOBM4BdJT0K7JQ8dz0o0E1+HeEPgy6rAfu/9vbKuzHT1gIOJqSXyKQi\ngEKVRe68LFJeFvXp1+Y2kr4GPGRmv6o4FgEnmtnkzILzlkFu4lhrA6tE0fIpyuNYK1Xn/5EYDFxM\nyBe0Z0+rip1zzdHwloGk6yXNIMwRXyRp++TnY8A7wGm1h+sGollzqONYO8SxYsJ04uq1IUy8jVWW\nj4shhNbCOsDOzagIfD55yssi5WVRn74GkPcDPk3ILrkeYfHYO1XvuSODuFx+XiCsNL85iuytyhdU\n1ihgtsra0kr2jsQIQrrpp4GpZsumHjvnWkx/u4lE6AveAjiuv2sL6uXdRNmJY40GLgSmdpMXqEcq\nazUr2YJl6SVgNjDDDE8d7VxBZDG1FAgry4Af1xSVy12y7SNVX/ovAucN9FxJRbAxcDNwOWHjepP0\nEGHXu0XANmbme18410L61TLIi7cMUrVMm4tjfYjQ1bcvsH8U2T/6fb2ytiPsX/F5K9myVOUSHyLM\nMDvNjAsr4psPjEiePmlm6w8k1oHwKYQpL4uUl0Wq4QPIruV9ltD6+zwwp78fUlmnEwaFr6MiGaHE\nBOAW4NjKiiDRNV7wOjChjpidcznwlkEbiGO9DxgdRfanRpxPZe0B3GUlWzYzKEkvcSnw2e7SS0ja\ngDB+MMG7iJzLV2ZjBq7wRgMN65axkv2+8rnUlV6CPcy4u9vPhAogs64h51y2vGXQIlZbTbvccAPv\nAraMIvt2o86rsiYDsZXszW5fF8eTcXqJgfK+4ZSXRcrLIuVjBm0qjjWkVOIXhF3nXu+aHdQgOxPW\nkCxHQhJn0YT0Es65/HnLoGDiWMOBd6o3fI9jjYoie7kZMXh6Cedam7cM2sOVwLbVBxtREaisd/f5\nnhzSSzjn8ueVQU7iWGvGcbdZXz8dRXZb9cF68q6orJVU1uHAQyprsx7fF9JL3ESYHjrFjNdrvWaW\nPAdNyssi5WVRH68MmiyO9eE41i3Av4FPVr9enRG0QY4j9P1HVrJHuntDkl7iNsJ6hAM9z5BzncXH\nDJosjrUBYW+Im6LI3mjGNVXWUOBtK9mSbl8P6SX+AFxBkl6iGXE557JRy3enVwYZSQaCfw5MiaLu\nv4SLoKf0Es651uUDyDmqnu4ZRbaQBu7e1t/+UJV1iMr6fP/O2Wt6icLyvuGUl0XKy6I+vgK5DnGs\nzUn3fDiK0Oe+TBTZ7BzCugP6HvjtK72Ec66zeDdRHeJYXwdGEqZi/iWjwd+Gq0gvMaWn9BLOudbl\nYwYZiWNtAmwYRXZL3rFUUlk7AvOtNIDU1AVML+GcaywfM8jOCGDDPAOo7A9VWaurrAuBnwKj+vf5\n9kkv4X3DKS+LlJdFfXzMIBHHWhmYCOwQRXZK5WtRZPcA9+QSWPfWTv7dwko2v6c3pbuPDV4E82bB\nqA2ACb6q2DlXzbuJgDjWIGAu8Ayh//+cIk8H7a+w+9iqI+AaYLW3YKd3F3VVsXOucXzMoB/iWEMB\nqhd8xbFGRpG90shrNYLKErCmlezFAX9WI1+A37wbnlsM33yf2YP/yiBE51zB+JhB/1xASNu8nCJW\nBIltgUsG2h8a0ks89zw8uhAO37SdKgLvG055WaS8LOrTtpVBHGtkHOs93bx0SBTZDU0PqEZWsj8D\nUwfymSS9xO2wyjVw2Opm8x/PJDjnXNtou26iZCHY2cD2hL7/mVnEVlSeXsI553sgBy8BlwFTk5QQ\nLUFlDQKOJswQOrSmc4T0EtcBR5txbSPjc861t5ZtGSRTQa8HPhlF9lZzI2sslbUK8H/AEuBQK9mj\nK7ynj/1dOym9hO91m/KySHlZpNp6ALmbRHDvAKcRvkBbmpXsbeDrhP0GVqgI+pKkl7gI2KPdKwLn\nXDYK3zKYNYtvEBLBlaLIfpt3TEXj6SWcc9XadcxgLWA6IRtny0u6hA4CLrFS7TWxhIAzgb0I6SWe\nalCIzrkOVPjKIIpset4xNNgSYCNgKP1INd2lsj9UYjBwMbA5HZhewvuGU14WKS+L+hS+Mmg3VrLF\nwMm1fl5iCCG/xCrALma0zIwp51xxFX7MoAgprOuhsja2kj3WkHOJEYQZVE8D03zTeudcdzw3UYGo\nrFHAOcAOhLUDb9Z1PjEGuAmYDcwwoyU20nHONV9LTi2V9LikOZLulfTXvONpoB8AC4CtGlARbAw3\n3UNYUDa90ysCz0GT8rJIeVnUpwhjBgZEZvZy3oE02EFWsnfqPUmaXuLBn5vtfmoD4nLOuRXk3k0k\naS6wtZmtMCOmlbuJGsHTSzjnatGS3USElsGtkv4m6bC8g6mFyrpUZW3X0HOKvQkVwQFeETjnslaE\nbqLtzexZSaOBWyQ9bGazu16UdDnwePJ0PnBfOt8+9BHm/ZyZfAd4rGHnw8YDp8Mh34DLFoMt1x+a\n9++b9/OuY0WJJ+fnW5rZuQWKJ8/nX6KA3w/NeJ48nkbwODXIvZuokqQSsNDMzk6ed1w3UU/pJXxB\nTcrLIuVlkfKySLXc1FJJQ4FBZrZA0jDgZqBsZjcnrxeqMlBZg4EZwFVWsmcbeu7l00tM8vQSzrla\ntWJuorWBXyskJB0M/KyrIigalbUFYZ+EV6Cxffidnl7COZe/XAeQzWyumW2Z/HzAzE7PM54+rEnY\nP3mSlezJRp00SS9xHbAOIb1EtxWBz6FOeVmkvCxSXhb1ybtl0DKsZDEQN/KcVeklpnp6CedcXgo1\ngFwtrzEDlTUMWL3R4wLLXcPTSzjnMtKq6wyK6AvAgVmdPKSX4HY8vYRzriC8MujehVay72Rx4iS9\nxJ+As8041Yx+Nc28PzTlZZHyskh5WdTHKwNAZS1XDvXsQNbrdUJ6iVuAY824MItrOOdcLTp6zEBl\njQb+F5hjpWxnMiXpJS4lpJfwTeudc5nxMYMBUFkbA/cDTwHfz/RaYhphHcGeXhE454qoYysD4N/A\nLlay461kb2R1kSS9xExgohl3134e7w/t4mWR8rJIeVnUp2PXGSTjAg9kdf6q9BI7eHoJ51yRdcSY\ngcpaF/iUley8BoTV9/WWTy+xp6eXcM41UyvmJmqW16E5c/mT9BLXAKsQ0kssbMZ1nXOuHh3RMmiW\nqvQS0xqZXsLT86a8LFJeFikvi5TPJgJU1rtU1oebft2QXuI2YA5woOcZcs61krZqGaisbQhz+edY\nyTJLJ7HCdUN6iT8AV0L/VxU751wWOnrMQGUNAs4BzgCubtp1Q3qJ3wOn+api51yrareWgbJKJdHt\n9UJ6ieuAo7PetN77Q1NeFikvi5SXRaqjxgyq8wlBdjmFur1+SC9xHSG9RKYVgXPOZa0lWwbJXsR3\nAbtZyV5sflxMA04HptSzqtg557JQS8ugJSsDCAvJrGRPNz8mjgeOAnY34+FmX9855/rSUd1Eza4I\nJCRxFnAwIb1EUysCz7uS8rJIeVmkvCzqU/jZRCprLPA14IQsE8r1GsPy6SUmeHoJ51y7aYWWwT+A\n+cCSPC6epJe4DliHkF4il4rAZ0mkvCxSXhYpL4v6FL5lAEyykt2Xx4Wr0ktM9VXFzrl2VfiWQY4V\nQaHSS3h/aMrLIuVlkfKyqE/hK4M8SGwE3A78Cphu1pyMp845l5eWnVqa3TU9vYRzrrV1dG6iRmhm\negnnnCsS7yZKFD29hPeHprwsUl4WKS+L+nhlwLL0EhcTtqi8NedwnHOu6Tp+zMDTSzjn2o2PGQyA\nhIAzgb0I6SWeyjkk55zLTUd2EyXpJS4FPk5IL1H4isD7Q1NeFikvi5SXRX06rmWQpJe4BliFkF5i\nYc4hOedc7jpqzKAqvcS0vFcVO+dcFjoqhfVAFS29hHPOFUmulYGk3SU9LOn/SToxu+u0fnoJ7w9N\neVmkvCxSXhb1ya0ykDQIOB/YnbBPwGclva/x1+FDwGzgbDO+aUZx+8V6t2XeARSIl0XKyyLlZVGH\nPFsGHwX+ZWaPm9k7hEHdTzTyAkl6iVuAY9sgz9AaeQdQIF4WKS+LlJdFHfKsDNYFnqx4/lRyrCGK\nnl7COeeKJM/KILPumiS9xI9or/QS4/MOoEDG5x1AgYzPO4ACGZ93AK0st6mlkrYFZprZ7snzk4Gl\nZnZmxXtatX/fOedyNdCppXlWBoOBR4CdgWeAvwKfNbOHcgnIOec6WG4rkM1ssaSjgT8Ag4BLvSJw\nzrl8FHoFsnPOueYo7ArkZi1IawWSHpc0R9K9kv6adzzNJOnHkuZJur/i2ChJt0h6VNLNkjpiSmEP\nZTFT0lPJvXGvpN3zjLFZJI2TNEvSg5IekDQ9Od5x90YvZTGge6OQLYNkQdojwC6EPEJ308HjCZLm\nAh8xs5fzjqXZJE0AFgJXmtkWybGzgBfN7KzkD4WRZnZSnnE2Qw9lUQIWmNk5uQbXZJLGAGPM7D5J\nw4F7gH2Ag+mwe6OXstiPAdwbRW0ZZL4grQVluslPUZnZbOCVqsNTgCuSx1cQbvy210NZQAfeG2b2\nnJndlzxeCDxEWKfUcfdGL2UBA7g3iloZZLogrQUZcKukv0k6LO9gCmBtM5uXPJ4HrJ1nMAVwjKR/\nSLq0E7pFqkkaD2wF3EWH3xsVZfGX5FC/742iVgbF67vK1/ZmthUwGTgq6S5wgIV+zk6+Xy4ENiTk\n5XkWODvfcJor6Ra5DphhZgsqX+u0eyMpi18SymIhA7w3iloZPA2Mq3g+Doq/G1lWzOzZ5N8XgF8T\nutE62byknxRJY4Hnc44nN2b2vCWAS+ige0PSyoSK4Cdm9pvkcEfeGxVl8dOushjovVHUyuBvwKaS\nxkt6F7A/YVOajiNpqKTVksfDgEnA/b1/qu1dD3whefwF4De9vLetJV94XT5Jh9wbkkTYuvafZnZu\nxUsdd2/0VBYDvTcKOZsIQNJk4FzSBWmn5xxSLiRtSGgNQFgk+LNOKgtJVwM7Au8m9AGfAvwWuBZY\nH3gc2M/M5ucVY7N0UxYlICJ0AxgwFzi8os+8bUnaAfgTYbOqri+xkwmZDDrq3uihLL4KfJYB3BuF\nrQycc841T1G7iZxzzjWRVwbOOee8MnDOOeeVgXPOObwycM45h1cGzjnn8MrAdShJZUlD8o7DuaLI\nbacz5xotWaC3TsWhfYEFwM0Vx14CPgysBXwgSYfdZU1grJmN7uH8Y4BhZvZYN69tAnzdzKYlz78L\n/MLM7uoj5k8T0pOf3M1rGwGvd8IiMpc/rwxcO1kDWC95LGAEsHLFMQh7bk8C9jazV4CJAJLWBG4A\npnZ34mTP7suBQ6qOnwFsD6wKjJM0O3lpPLCLpAXAw4QVw7t1c+qRwFBJuwB/M7P/qXjtTeAKSXua\n2ZK+fnnn6uErkF1bkfRF4FDCJjDDCEvx3wCGEvbFGAecCVwAnGJmj0j6GPAz4Coz+1oP5z0Q2MDM\nvtXD62sBnyLkkh8FDAHuqmxFJDlkVjGztyqO7UtoGXy1h/N+FXjKzK7sfyk4N3DeMnDtZjgwi5DP\nvXJjjw8Cq5vZCZJ2BV4Dhkn6GeGLey9guqTfEVIAV3cFfQY4prsLSjoN2BOYT+h+Ggw8Axwr6S9m\n1vW5Sck1plT9pd/bBiRXAz8AvDJwmfIBZNeOliQ/i5OfJcBSAEnrELp7riF0+ZxvZp8yswfN7HBC\nzvcvdnPOjcxsbi/X/JKZTQROB76XPF5uIyIz+wMhYdiP+vuLJNfcsL/vd65W3jJw7UbA3sC2VcdH\nAb8CfkLYEWsxoQXx7dB7w1jCBiAAx/dw3hUPSucRtlqcIulVkp21JH2O0E21tqQ1zOzzyUdmAL+T\ntLmZ/bOm39C5DHhl4NqNETb5uIPlv8C3AlYh7I1xFICZXZe8F0k3mtnkXs67tNuLhS6gY5JzrE/o\nznkFOMbMVtiQKeke6u06y5E0iNCycS5TXhm4diPCX+Qjk8eW/LsasMjMXkxaAgP1d0lbm9nfVrhg\n2HToEEL++C8AbwM/kfRz4Jo+8umvSxjs7snWwN21BOzcQHhl4NqGpAuAnYBFhOmelYYTBow/BDyQ\nvH9WxevDKp7PNbNDqj5/EXAEYaZS5TVPBPYDrgB2NLN3kuO7AQcDv04GkU+u+MyXgRMIs5zeJnRr\n9eTw5NrOZcqnljrXT5J+SNhj9vaKY6uY2dsZXW9HYH8zOzKL8ztXySsD55xzPrXUOeecVwbOOefw\nysA55xxeGTjnnMMrA+ecc3hl4JxzDq8MnHPOAf8fCgn2gtZjZhgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x79b6160>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt = runplt()\n",
    "plt.plot(X, y, 'k.')\n",
    "y3 = [14.25, 14.25, 14.25, 14.25]\n",
    "y4 = y2 * 0.5 + 5\n",
    "model.fit(X[1:-1], y[1:-1])\n",
    "y5 = model.predict(X2)\n",
    "plt.plot(X, y, 'k.')\n",
    "plt.plot(X2, y2, 'g-.')\n",
    "plt.plot(X2, y3, 'r-.')\n",
    "plt.plot(X2, y4, 'y-.')\n",
    "plt.plot(X2, y5, 'o-')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "成本函数（cost function）也叫损失函数（loss function），用来定义模型与观测值的误差。模型预测的价格与训练集数据的差异称为残差（residuals）或训练误差（training errors）。后面我们会用模型计算测试集，那时模型预测的价格与测试集数据的差异称为预测误差（prediction errors）或训练误差（test errors）。\n",
    "\n",
    "模型的残差是训练样本点与线性回归模型的纵向距离，如下图所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEaCAYAAADzDTuZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe8XFW5//HPl9BDhwChBhQVDVWiSJFgoShyVRTFhlIM\nUhUUAio7226UIlKkSpHmFa+CiqJeUMBCDYi0qxKFHyWoRClCIHl+f6x9sieHc3LKnJm9Z+b7fr3y\nOlP3fs7DMM9Za+21liICMzPrbUtUHYCZmVXPxcDMzFwMzMzMxcDMzHAxMDMzXAzMzAwXA7NFSGrb\n/xOScknLjeJ9S7YiHutt/lBZLUhaEbgeWNzEl1cBNxa3twJuH+D2fRFxkKRlge9GxJ7F8Q8GDgTe\nEBFPNJz3A8C4iLhA0vLA7yV9HphfvOSGiHiseO2tEfHqxfwOGwHrNDy0F/AkcE3DY/+IiHslvQ9Y\nE5gsaWbD86sDEyNigqSNgdcA44G5wNrAlUAm6RsR8YeGc38BuArYHdijOG+ficAxEfHDwWI3czGw\nuhgHPBERO0vaGdglIo6VdHNETOn/YklHR8TM4vapEXFov5e8C/hl8fxuwCuAfYALJb0/Iv4t6XTg\nDeklWgdYFfgDcAOwMnApsGtxjHHAJpJ+O0j8dwBnAuv1hVgcY6mGx/pi3wU4BNijKEw7F4+vTvpC\nf3fx0gdIBeUJYEVghYh4UNLrgGslbU4qEqsC25MKybLAtIi4teF8+w8Ss9lCLgZWR8sAz/d/UNIm\nwBGk1sMdDU+tJmm9iHioeJ2Aw4BjJH0aWBc4NCIWSMqAqySdExEHF62A/wfcSWphzCN9Ge8OHNnX\nKiiO8euI2GNxgUvaGjgAeIr0F32QCtHywKURcbmkrwF7At+SdHxE3CdpO+Bi4JKIuK443LHAm4u4\nHgIel/Qv4L7i+IcCFwH3AA8XP18KnCapsWWwOvDpxcVt5mJgdbQSi3Zz9FmX9BfyQ8DLJJ0KbAv8\nE7hE0krACZR/ja9A+oJ8BNiMVEA+CPS1FADeTuoS+hpwOvB+4MukAjFPkiKt2TIB2FbSzYPEvH1E\nzCvOeS3wO1LroM/mpJYCEfEpSW8G/g2Ml3QxsBype+dwST8iFb2vkP4fXar4+RVgOvBYRPxQ0uHA\nD4pWzl+A35NaCYdExK2SjgF+GhGNhdNsQC4GVkevIP2lvlDR1bMZcDfwRtKX9lzgXGAaMAO4NiJ+\nKukk4Nukbqcbi/70+4tDbQk8FxG3F8d8AjgH2ITUrXQ16a/rDYFPAp8A9iq6XdYYKNhiLGFew0Pz\nKcccIBWF+cVPii6p84EPAfsBp0ZEX/fTNElTgf2BF0hf8H8H/gzsBBwJfKMYRF6+KATrAa8kFbZ7\ngLdKehqYBLy7aCXcFxEHDRS/GbgYWM1IWhN4B/D14qHni770LUlf3KuQCsB+pC6UIA2qAmwtaUfS\nF+bxDYddFfjXAKf7EHA56Uv6DNKA7p7Fvz57jfRXAN5GarE0Wg24orj9HdKX/AukVsSXUs8WE0mt\nGICjgfcVv8dGwB+BJyPiOEl3k1pAfQXkM6RCdQSpi+vIiHio6BL7UeP4gdlgXAysLoJ0qfNFwBER\n8Uzx+KnAj0h/HZ9P6ip6jPSFvSAidhnoYMWXa9+VSVtQtgwaHUUaQF4yIh6TtGlE7NhwjG8Dy0n6\nPrD+IDHnLNodFKQv/Rv7Pb4VaSwE4D3AwQARcUXxeiRdHRG7N5z/XtIVVhsV5z+meOp0YBbw8uIY\nBxVjH08Xr5vW0A21g6QNgI9GxK8G+B3MABcDqw8BC4DdI2JB34MRcQlwCUDRfbI06Qv8+5RX3Qx6\nzOI9D0fEc/2fjIhHiqKxWBHxzsWc4OXAo/1+j/Gk1ohIxUGksY55xfEeH+Z5n5R0LKn7agKwSzEP\n4hjSmMQnJB1SjGksQRqTeJbU4jikr0Ug6aghT2Y9r+XFQNL6wIWkJngAZ0XEKZJmkK66eLx46bER\n8dNWx2O19QKpC+jWxXxRLksaCziJdAnmAcM47jGkyzj7zOPFcxkW3pd0fcPjLwWy/geUlJP+un+O\ndJXQocXjfZeqziNd6tloBdJg8eYR8Z6GY13b8JrxDfcfII1fvA/4TfHvCOBlwG8i4v2SzgC+J+mv\npC4zAaeQLlVtvKJofWCwgW8zANTqzW0krQ2sHRGzJK0A3Eoa6Nqb1Ad6YksDMOtQkqYA90bEkw2P\nvSYibmq4v0FE/K243Xflk9mItbxlEBGPUjSjI+IpSfeQ+n1h0T5VM2sQES/6a76xEBT3/9Zw24XA\nRq2taxNJmkQaSPtd8dBhku6QdK6kVdoZi5mZldq5KNcKwPdIV4o8RbqUbyPSJYOPkC6VMzOzCrR8\nzABA0lKkywOvjoiTB3h+EnBVRGzW73E3e83MRiEiRtQN346riUSaJHR3YyGQNDEi+ibYvIO0QNiL\njPQX6laSZkTEjKrjqAPnouRclDopF8o1jjS7fTrwReCUyGL+4t81guOP4g/pdswz2B74AHCnpL5l\nho8D9pG0JemyvgdISwrY4CZVHUCNTKo6gBqZVHUANTKp6gCGQ7kmA+eRFht8bWTx54pDAtpzNdEN\nDDw2cXWrz21mVhfKtTRpJdpDSX8QnxNZfa4A8wzkznF+1QHUyPlVB1Aj51cdQI2cX3UAg1GuKaTW\nwGxgq8jScut10pYB5NEq5tB4zMDMOpJyLQ98jtRV/gngsna0Bkbz3ek9kDtEscaO4Vw0ci5KdcuF\nck0l7aGxDrBZZHFpnbqF+nM3kZnZGFKulYCZpM2KPhZZXFVxSMPibiIzszGiXG8BvgX8FDg6sphb\nSRyj+O50y8DMrEnKtQZwMvA64MORxf9WHNKIecygQ9StP7RKzkXJuShVkQvlknK9hzRpdg6weScW\nAnDLwMxsVJRrHdKuc5sA74gsfjfEW2rNYwZmZiOgXCLtwf1l0vjAFyN78U56VfKYgZlZCynXRsDZ\npF353hRZ3FlxSGPGYwYdwn3DJeei5FyUWpkL5RqnXEeQtg+9Bti2mwoBuGVgZrZYyrUpaeXlF4Dt\nIov7Kw6pJTxmYGY2AOVaCjga+DhwPHBmZLGg2qiGx2MGZmZjQLm2Ji0s9wjw6sjKvaa7lccMOoT7\nhkvORcm5KI1FLpRrOeX6MmmJ/ROAt/RCIQC3DMzMAFCuHUhjA3eSJo89VnFIbeUxAzPracq1IvAl\n4J3AYZHF9ysOqWlewtrMbASUa1fSUhLjgcndUAhGy91EHULS1Ii4ruo46sC5KDkXpZHkQrlWA04E\npgIfjSyuaWFoHcEtAzPrKcr1TuAu4N+k1kDPFwLwmIGZ9QjlWhs4FZgMHBBZ3FBxSC3jeQZmZv0U\nC8t9CPgacA7wgcji2Wqjqh93E3UIX09eci5KzkVpoFwo1wbAT0iziHeLLI5zIRiYi4GZdR3lWkK5\nDgFuBa4HXhNZ3FZxWLXmMQMz6yrK9TLS5LElgP0ji3srDqntPGZgZj1LuZYEjgI+BXwOOC2ymF9t\nVJ3D3UQdwn3DJeei5FwkyrUF9/NH4E3AlMjiFBeCkXHLwMw6lnItA3wGmMYcvs3LmB5Zjfu+a8xj\nBmbWkZRrW9Iy0/cBh0QWD1ccUm14zMDMup5yjQe+ALwXOBz4nlsDzfOYQYdw33DJuSj1Wi6U642k\nheXWIC0l8d99haDXcjHW3DIws9pTrlVIM4h3BQ6KLH5ScUhdx2MGZlZryrUncDpwJTA9svh3xSHV\nnscMzKxrKNcE4BRgG+D9kcWvKg6pq3nMoEO4P7TkXJS6MRfKJeV6H2ls4CFgi+EUgm7MRTu5ZWBm\ntaFc6wFnABsCb4ssbq44pJ7R8jEDSesDFwJrAgGcFRGnSFoNuJz0H302sHdEzO33Xo8ZmPUA5VoC\nOAD4IvBN4CuRxbxqo+pco/nubEcxWBtYOyJmSVqBtIrg24GPAH+PiJmSjgFWjYjp/d7rYmDW5ZTr\nJaR9BpYnLSx3V8UhdbzRfHe2fMwgIh6NiFnF7aeAe4B1gT2BC4qXXUAqEDYI94eWnItSJ+dCucYp\n15HA74GrgO2aKQSdnIs6aOuYgaRJwFak//hrRcRjxVOPAWu1MxYzq45yTSYtM/0MsG1k8aeKQ+p5\nbSsGRRfRFcAREfGkVLZgIiIkDdhfJel80pgCwFxgVkRcVzw3tXh/19+PiOvqFI/v1+d+n7rEs9j7\ny7Ik09keOJQbuYBf8uOYnwpBs8fve6xWv2+b7he3P1ykYTaj0JZJZ5KWAn4EXB0RJxeP3QtMjYhH\nJU0Ero2IV/R7n8cMzLqEck0htQb+RppF/FDFIXWtWo4ZKDUBzgXu7isEhSuBfYvb+wI/aHUsncz9\noSXnotQJuVCu5ZVrJmlc4CukS0bHvBB0Qi7qrB3dRNsDHwDulHR78dixpA/FdyXtT3FpaRtiMbM2\nUq6dSFcK3QJsHlnMqTgkG4TXJjKzMadcKwFfBd4WM1gX/3/cVrXsJjKz3qJcbyEtJbEkMLnicGyY\n3DLoEI1XSfQ656JUp1wo1+rAyaSu4QMji1+mJxTtaBnUKRdV86qlZtZ2yiXg3cA3gMuAzSKLp6uN\nykbKLQMzGzXlmkjaa+DlpKUkfvviF7WnZWAltwzMrC2K1sBHSFcFngm8N7J4rtqorBkuBh3C/aEl\n56JURS6UaxJwFrA68ObI4o52nn8w/lw0x1cTmdmwFAvLHU6aM/BL4LV1KQTWPI8ZmNmQlGtT0uSx\nBaSxgfuH/2aPGbSbxwzMbEwp11LAp4AjgQw4I7JYUG1U1gruJuoQXnel5FyUWpkL5doKuAl4PfDq\nyOK0OhcCfy6a45aBmS1CuZYFjgf2J7UKLoqsxv3JNiY8ZmBmCynX9qRVhu8CDo0sHm3+oB4zaDeP\nGZjZqCjXCsCXgHcBh0UWV4zJcaV7Iv2cA0yJiL+OxXFt7HnMoEO4P7TkXJTGIhfKtQtpYbkVgclj\nVQgKE4ufE4Drx/C4L+LPRXPcMjDrUcq1KnAisDMwLbL4WQtOM6/oq3ga2LEFx7cx4jEDsx6kXO8A\nTgW+DxwXWTzZkvNIG5JaBDu6i6h9RvPd6WJg1kOUay1SEdicNHnshopDshbw5jZdzP2hpW7MhaR7\nJM2VNKf4a3q475s6rNflknJ9ELgT+BOwZbcVgm78XLSTxwzM6mFiwMrFn3LXAxuM1YGVawPgW8A6\nwFsii1vH6tjWPdxNZFYDkuYETFAaaH3VWPSvK9cSwDTgc6QdyGZGFs83e1yrP88zMOtcU4DZjF0h\n2IS0sNzSwE6Rxd3NHtO6m8cMOoT7Q0vdmIu+AjDSQtA/F8q1pHJ9Cvgt6UqhHXqlEHTj56Kd3DIw\n6xLKtTlpKYl/AVMiiwcqDsk6iMcMzOpilGv4KNcywKeBjwHTgfO8sFxv85iBWY9RrtcC5wH/B2wR\nWTxccUjWoTxm0CHcH1pyLkC5xivXifyZn5CuFnpHrxcCfy6a45aBWYdRrjcAZwO/5efsFxfGD6uO\nyTqfxwzM6mKIMQPlWhn4GrAb8LHI4sdti806ipejMOtSyvU20oYzC0jLTLsQ2JhyN1GHkDQ1Iq6r\nOo466KVcKNcE4BvAa4APRrbo791LuRiKc9EcFwOzGlIuAe8FTgIuAjaPLJ6pNirrZh4zMKuLYsxA\nudYFzgA2Ii0zfVPFkVmH8ZiBWYdTrgOBWcBtwKtdCKxdXAw6hK+hLnVjLpRr4+LmgcAbIosZkcW8\nId/XhbkYLeeiOR4zMKuQco0DDictJwGwXWTxQoUhWY9qy5iBpPOAtwJzImKz4rEZwAHA48XLjo2I\nn/Z7n8cMrGsp16tIC8s9CxwQWfyp4pCsS9R2D2RJOwJPARc2FIMMeDIiTlzM+1wMrOso19LAMaQW\nwWeAsyOLBdVGZd2kJQPIksZJ+pCknxT7tN4j6S5J35a07XBOEhHXA08MdPiRBNvL3B9a6uRcKNc2\nwM3AtsDWkcWZzRSCTs7FWHMumrPYYiBpLeDHwHrAIRGxaURsCmxB2lP1o5JOb+L8h0m6Q9K5klZp\n4jhmtaZcyynXTNL/TzOBPSKLBysOy2yhxXYTSdoY+E9EPLKY1+wUEb8a8kTSJOCqhm6iNSnHCz4P\nTIyI/fu9x91E1vGU6/WkLShvBw6LLOZUHJJ1uTHfzyAi/jLUAYZTCAZ538L/ISSdA1w10OsknU/a\nGxZgLjCrb8p5X7PQ932/lvdX11vYiQPZginAocxgLvBKMubUIj7f75r7xe0Pk8xmFNo2A3mAlsHE\nvhaHpE8AUyLiff3e45ZBweuulDohF8q1O6kr9efAJyOLuS05Twfkol2ci9KYtwzGiqRLgZ2ANSQ9\nCGTAVElbAgE8AExrRyxmraRcq5PWE9qRtJTELyoOyWxYht0ykHRaRBxS3J4JTAHOiYiLWxacWwbW\nIYqF5fYCTgG+C3wmsniq2qisV435PANJ34mIDxS3fxsRr2t4biPgpxHx8tEGPGRwLgbWAZRrInAa\nsCmpNfCbikOyHteKeQYvGeyJiHgA+PtITmaj52uoS3XJhXJJuT4C3AHcDWzV7kJQl1zUgXPRnJGM\nGSwr6b/7vXf8GMdj1hGUaxJwFrAGsEtkMavSgMyaNFQ30cKuIUm/A/aFhbOGA/hbRPynZcG5m8hq\nRrmWAA4hXQTxdeCEyOL5aqMyW1SrryaKiLhvhDGZdQ3lejlpYTmA7SPz/w/WPUayn8HySmsS/UbS\nycW8AWsT94eW2p0L5VpKuY4FbgQuA15fl0Lgz0XJuWjOSFoGz0TE64plJPYArpH03oi4rUWxmVVO\nubYitQYeB7aJLGZXG5FZaww1ZtA4t6D/paWbA5cAW0cMvSPTqILzmIFVRLmWBT5L2nnsaOCCyGq8\nYbhZgzEfM+grBIVf9XvuTqUVS9cAHh7JSc3qTLm2I7UG7gY2jywerTgks5Ybzn4Gn5S0JHC8pFOL\nxz5fPP0QMLGF8VnB/aGlVuVCuVZQrm8AVwCfjSz2qnsh8Oei5Fw0ZzhjBm8F/gLMB15VPLZD8XNn\n4MoWxGXWVsr1ZtK8gV8DkyOLf1QckllbDVoMJL0CeGnfXVh0VzJJAl5D6k+1FvNqjKWxzIVyrUqa\nL/Am4KDI4uqxOnY7+HNRci6as7huogeBo4ClSBPM+g+efRy4IMITbqwzKdfbgbtIG9JP7rRCYDaW\nBm0ZRMTTknYn7de6O7AAmCDpvcBWwI0RcVJ7wjSv1Z5IugdYH3iGtAfGX0d8jFxrAd8EtgT2iSx+\nPbZRto8/FyXnojmLHUCOiGdJReBZ4Lni9jOkIrKepJFMWjMbCxOvTWtiTQCuH8kbi4XlPgDcSdpD\nY4tOLgRmY2nQeQaSliPNtlyZ9FfUfOCIiNhZ0rXApcCGEfHplgXneQbWj6Q5ARMETwOvGm7LQLnW\nB84E1iUtM31LK+M0q9JYL2E9mbRjUzDAmEFEnAW8XtLKIw3UrAlTip/DKgTKtYRyHQTcBvwWmOJC\nYPZigxaDiLi5of9tQfGvvx8Bb25BXNaPr6FOIuKv1xU/h3qtcm0C/C9po/CpkcXnI2vNbPmq+HNR\nci6aM5x5BpdHxA8AJPV98f+0+HkdqRvJrDaUa0ngE8AxwBeBUyKL+dVGZVZvw94DuQoeM7ABScEg\nnwvl2gw4D/g3cGBk8Ze2xmZWA63ez8CstpRrGeA44GDgWOBcLyxnNny+NLRDuD+0dF2/+8r1WuBW\n0vyXLSOLc3qlEPhzUXIumrPYloGklwD/iYhBVyWVtENE3DDmkZkNQbmWBz4PvJ80I/7yXikCZmNt\nqP0M1gbOJ03uuSQiHmh4bhvgIOCFiDioJcF5zMAGIoVm8AbgbOAm4IjI4vGKozKrjVbsZ/CopD2A\n9wLflLQxacG6BaTrts90q8DaSblWLv58uRA4OLK4qtKAzLrEkAPIEfEC8J3in1XE666Acu0BnHFd\nujs5svhXlfHUgT8XJeeiOb6ayGpPuSYAJwPbAvvuDAtcCMzGlucZWG0pl4D3kArBxcDxkcXT1UZl\nVn+eZ2BdQ7nWBU4nbbD0X5HF7ysOyayrLXaegaS1JK1V3JakKyStKWnD4t8G7QnTeuUa6mKZ6QOA\nWcAdwNb9C0Gv5GI4nIuSc9GcoVoGOwCflXQkMA/4B2mz8D8Vz+8GTGxdeNZLlGtj0uWiKwNvjCzu\nrDgks54x1DyDvYC7SUtZrwJ8CDg3InYsnr++73ZLgvOYQU9QrnHAYcBngK8CJ0UWL1QblVnnasWY\nwTjgX8DVwCeBEW8xaLY4yvVK4FzgeWC7yOL+ikMy60mDjhkUW1p+Fvg6aYvBY4HD2xSX9dNt/aHK\ntZRyfRb4NWkC2dThFoJuy0UznIuSc9GcQVsGEbFA0muBGcAjpEv7fgn8odj2EsBNeRsx5Xo1aZnp\nh0kDxH+rOCSznjfkPANJxwK/AjYCNgHOiIjH2hCbxwy6jHItB2TAR0jdjt/xwnJmY2/Mxwwk7Uva\nMvAJ0qJ0JwK7SQvPERFx4TACOw94KzAnIjYrHlsNuBzYEJgN7B0Rc0cSvHUO5dqRNDYwC9g8svb8\nQWFmwzPUfgbzgQDWJQ0mHwCsQOoeeqF4fji+TboMtdF04OcR8TJS99P0YR6rJ3Vqf6hyrahcpwGX\nAcdEFns3Wwg6NRet4FyUnIvmDLVq6XckbQfcR7qsdK3i51ER8chwTxIR10ua1O/hPYGditsXkPYs\ncUHoIsq1K3AWqdhPjiyeqDgkMxvEcMYMVgKuIv1lP694uO9NEcNc3KgoBlc1dBM9ERGrFrcF/LPv\nfsN7PGbQgZRrNdLclJ2Aj0YW11QckllPacWYwaqky/4+AXwJ2LLfS+YBu47khAOJiJDkgcQuoFzv\nAk4BvkdqDTxVcUhmNgxDTTr7X2CfiLiXtJnNWHpM0trFBjoTgTkDvUjS+aQBZoC5wKy+Ncv7+gh7\n4X5jf2gd4nlRfLnW5o9cztJsxCa8O7K4UdJUzdCYn69/Turw+1d4f8uIOLlG8VR5/+P09vfDh0lm\nMwpDLUdxMPAK0v6ymwErkmYk3x8Rz43oRC/uJpoJ/CMivippOrBKREzv9x53ExVU0407imWm9wVm\nAucAn4ssnm3pOWuaiyo4FyXnojSa787hjBnsRhrsvQXYGFgV2II0YWh6RPxlGIFdSuo/XgN4DDge\n+CHwXWADBrm01MWg3pRrQ+BM0oUF+8UMbsP/vcwq15JiUBz408A9EfH9hsemAsdExO4jDXTYwbkY\n1JJyLQEcTJqdfiLwtcjieaRwMTCr3pgXA0lXki4LvIM0v6Dv0sAABBARN44q2uEE52KwUF2awMr1\nclJ30BLA/pHFveWT7SkGdclFHTgXJeeiNOZXEwF7A+8iLSu8Hmny2PP9XtOyYmD1oVxLkpaQ+CSQ\nA6dHFsOddGhmNTfcbiKR1pPZDDgyhvOmMeCWQT0o15akpST+CRwYWcwe+IXuJjKrg5aNGVTFxaBa\nyrUsacOZacAxwLcHW1hO0j0BrxA8DkyJCO99YVaR0Xx3DrU2kdVE4zX2bTlfrtcBtwOvAraILM4b\nYoXRvu1PJwDXtzQ2r0GzkHNRci6aM9SYgfUY5RoPfBF4D2kzo+8Nc5npecWfIU8DLdsK1cxaw91E\ntpByvYm0sNyNwMcji38M+73ShqQWwY7uIjKrlscMbFSUaxXgBODNwEGRxU8qDsnMmuAxgy7Wqv5Q\n5fov4C7gOdLCcrUvBO4bLjkXJeeiOR4z6FHKtSZpddFXA++PLH5VcUhmViF3E/WYYmG595GWkbgA\nmBFZPFNtVGY2lloxA9m6iHKtD5xBWhxwj8ji5opDMrOa8JhBh2imP1S5llCuaaQ9KW4CtunkQuC+\n4ZJzUXIumuOWQZdTrpcCZwPLAztHFndVHJKZ1ZDHDLqUco0jbVc6nbRl6Te8sJxZb/A8AwNAuSYD\n5wFPxwymevE4s97ieQZdbDj9ocq1tHJlwLWkPQfe2Oq4quC+4ZJzUXIumuMxgy6hXFNIrYHZwFaR\nxUMAzHCjwMyG5m6iDqdcywOfAz5IGiO4dJGF5bzHgFnP8TyDHqNcU0lXCt1CWkri8WojMrNO5WLQ\nIRr3d1WulYCZwB7AwZHFlVXG1m7e67bkXJSci+Z4ALnDKNdbSAvLjSO1BgYtBJLuKX7OKZaYNjMb\nkMcMOoRyrQGcDLwO+Ghk8csh3yPNDVi5SOCDEbFBS4M0s1rwmEEXKhaW25tUCC4DNo8snh7m2737\nmJkNi1sGNaZc6wCnAy/jSk6NW+P0Eb2/S3cfc99wybkoORclTzrrEsol5dofmAXcCWzFbdw90uNE\nxF8jYoNuKgRm1hpuGdSMcm1Eulx0FWD/yOKOikMysw7jlkEHU65xynUEcDNwDbCtC4GZtYuLQQ0o\n16akvv13AdtFFjMjixcWeY3XXVnIuSg5FyXnojm+mqhCyrUUcDRpGYnjgW9FFguqjcrMepHHDCqi\nXFuTFpZ7FJgWmQd5zWxseJ5BB1CuZYEM2B/4JHDRIgvLmZlVwGMGbaRcOwB3AC8lTR67cLiFwP2h\nJeei5FyUnIvmuGXQBsq1Imnryb2AQyOL71cckpnZIjxm0GLKtQtwFmn3saMii39WHJKZdTmPGdSI\ncq0GnADsTBog/lnFIZmZDaryMQNJsyXdKel2STdVHc9YUK53kpaZfgrYbCwKgftDS85FybkoORfN\nqUPLIICpEZ3ffaJcawOnApsBe0cWN1QckpnZsFQ+ZiDpAWCbiPjHAM91xJhBscz0B4GvkeYO5JHF\ns9VGZWa9qlPHDAL4haT5wJkRcXbVAY2Ecm0AnAlMBHaPLG6rOCQzsxGrQ8tgYkQ8ImkC8HPgsIi4\nvngugAuA2cXL5wKzFu4FXPQRVnFfuZbgN5zA2uzHxswEZjKD7Vt2vob+0Dr8/lXe75+TquOp+P6W\nEXFyjeKp8v7Hqcn3Q7vvF7c/TDIbyEbaMqi8GDSSlAFPRcQJxf1adhMp18uAc0gtq/0ji3tafk5v\n3LGQc1FyLkrORWk0352VFgNJywPjIuJJSeNJSzfnEXFN8XytioFyLQkcBXwK+DxwamQxv9qozMwW\n1YljBmt1Ay55AAAJcklEQVQB/yOpL5aL+wpB3SjXFsC5pK6qKZHFAxWHZGY2ZmrVTdRfHVoGyrUM\n8BlgGjAd+HYVC8u5CVxyLkrORcm5KHViy6DWlGtb0qWi9wNbRhYPVxySmVlLuGUw0HlzjQe+AOwD\nHAF818tMm1mncMtgDCjXG0gb0v8GmBxZ/L3ikMzMWs7FoKBcq5BmEO8KfCyy+HHFIS3C/aEl56Lk\nXJSci+ZUvlBdHSjXnqSF5eaTWgO1KgRmZq3W02MGyjUBOAWYAhwQmf+qMLPON5rvzp5sGSiXlOt9\nwB+Ah0hbUF5XbVRmZtXpuWKgXOsBVwLHAntGFp+KLJ6pOKwhea32knNRci5KzkVzemYAuVhm+kDg\ni6Q9B/aKLOZVG5WZWT30xJiBcr2EdLnoCsB+kcVdTQdnZlZTnmfQj3KNI00aOw74MnCyF5YzM3ux\nrh0zUK5XkSaO7QlsG1mc0MmFwP2hJeei5FyUnIvmdF3LQLmWJi0odxhpgbmzI4sF1UZlZlZvXTVm\noFxTSMtMPwgcFFk82LLgzMxqqmfHDJRrOSAH9gWOBC7xwnJmZsPX8WMGyrUTcCewAbBZZHFxNxYC\n94eWnIuSc1FyLprTsS0D5VoJ+CppgPjgyOKHFYdkZtaxOnLMQLl2B84k7Zn8ychibtuDMzOrqa4f\nM1Cu1YGTgB1Ik8d+UXFIZmZdoSPGDIqF5fYmLTP9T9LYQE8VAveHlpyLknNRci6aU/uWgXJNBE4H\nXkFaT+g3FYdkZtZ1aj9mwAweB84CvhBZPFt1TGZmddetYwa7RBazqg7CzKyb1X7MwIUgcX9oybko\nORcl56I5tS8GZmbWerUfM2jlHshmZt3IeyCbmdmouBh0CPeHlpyLknNRci6a42JgZmYeMzAz6zYe\nMzAzs1FxMegQ7g8tORcl56LkXDTHxcDMzDxmYGbWbTxmYGZmo1JpMZC0m6R7Jf2fpGOqjKXu3B9a\nci5KzkXJuWhOZcVA0jjgVGA34JXAPpI2rSqeDrBl1QHUiHNRci5KzkUTqmwZvAb4U0TMjojngcuA\n/6ownrpbpeoAasS5KDkXJeeiCVUWg3WBBxvuP1Q8ZmZmbVZlMajvZUz1NKnqAGpkUtUB1MikqgOo\nkUlVB9DJqtzp7P8B6zfcX5/UOliEJBeNgqR9q46hLpyLknNRci5Gr7J5BpKWBO4D3gg8DNwE7BMR\n91QSkJlZD6usZRARL0g6FPgZMA4414XAzKwatZ6BbGZm7VHbGciekFaSNFvSnZJul3RT1fG0k6Tz\nJD0m6Q8Nj60m6eeS7pd0jaSeuKRwkFzMkPRQ8dm4XdJuVcbYLpLWl3StpD9KukvS4cXjPffZWEwu\nRvTZqGXLoJiQdh/wJtJA88308HiCpAeAV0fEP6uOpd0k7Qg8BVwYEZsVj80E/h4RM4s/FFaNiOlV\nxtkOg+QiA56MiBMrDa7NJK0NrB0RsyStANwKvB34CD322VhMLvZmBJ+NurYMPCHtxXpywb6IuB54\not/DewIXFLcvIH3wu94guYAe/GxExKMRMau4/RRwD2meUs99NhaTCxjBZ6OuxcAT0hYVwC8k3SLp\nwKqDqYG1IuKx4vZjwFpVBlMDh0m6Q9K5vdAt0p+kScBWwO/p8c9GQy5+Vzw07M9GXYtB/fquqrV9\nRGwF7A4cUnQXGBCpn7OXPy9nABuR1uV5BDih2nDaq+gWuQI4IiKebHyu1z4bRS6+R8rFU4zws1HX\nYjCsCWm9IiIeKX4+DvwPqRutlz1W9JMiaSIwp+J4KhMRc6IAnEMPfTYkLUUqBBdFxA+Kh3vys9GQ\ni+/05WKkn426FoNbgE0kTZK0NPAe4MqKY6qEpOUlrVjcHg/sAvxh8e/qelcCfTNN9wV+sJjXdrXi\nC6/PO+iRz4YkAecCd0fEyQ1P9dxnY7BcjPSzUcuriQAk7Q6cTDkh7csVh1QJSRuRWgOQJgle3Eu5\nkHQpsBOwBqkP+Hjgh8B3gQ2A2cDeETG3qhjbZYBcZMBUUjdAAA8A0xr6zLuWpB2AXwN3UnYFHUta\nyaCnPhuD5OI4YB9G8NmobTEwM7P2qWs3kZmZtZGLgZmZuRiYmZmLgZmZ4WJgZma4GJiZGS4G1qMk\n5ZKWqzoOs7qocg9kszFVTNBbp+GhvYAngWsaHvsHsDWwJjC5WA67z+rAxIiYMMjx1wbGR8SfB3ju\npcBnIuLDxf2vA/8dEb8fIuZ3kZYnP3aA5zYGnu6FSWRWPRcD6yarAOsVtwWsDCzV8BikPbd3Ad4W\nEU8AOwNIWh24Cnj3QAcu9uw+H9iv3+NfAbYHlgXWl3R98dQk4E2SngTuJc0Y3nWAQ68KLC/pTcAt\nEfGxhuf+A1wg6a0RMX+oX96sGZ6BbF1F0v7AAaRNYMaTpuI/AyxP2hdjfeCrwOnA8RFxn6TtgIuB\nSyLi04Mc9wPAhhHxxUGeXxN4J2kt+dWA5YDfN7YiijVklomIZxse24vUMjhukOMeBzwUERcOPwtm\nI+eWgXWbFYBrSeu5N27ssTmwUkQcLenNwL+B8ZIuJn1x7wEcLulHpCWA+3cFvRc4bKATSvoC8FZg\nLqn7aUngYeATkn4XEX3v26U4x579/tJf3AYklwKnAS4G1lIeQLZuNL/490Lxbz6wAEDSOqTunstI\nXT6nRsQ7I+KPETGNtOb7/gMcc+OIeGAx5/x4ROwMfBk4qbi9yEZEEfEz0oJhZw/3FynOudFwX282\nWm4ZWLcR8DZg236PrwZ8H7iItCPWC6QWxJdS7w0TSRuAAHxqkOO++EHpm6StFveU9C+KnbUkvZ/U\nTbWWpFUi4oPFW44AfiTplRFx96h+Q7MWcDGwbhOkTT5uZNEv8K2AZUh7YxwCEBFXFK9F0tURsfti\njrtgwJOlLqDDimNsQOrOeQI4LCJetCFT0T20uPMsQtI4UsvGrKVcDKzbiPQX+arF7Sh+rgjMi4i/\nFy2BkbpN0jYRccuLTpg2HdqPtH78vsBzwEWSLgcuG2I9/XVJg92D2Qa4eTQBm42Ei4F1DUmnA28A\n5pEu92y0AmnAeAvgruL11zY8P77h/gMRsV+/958JHES6UqnxnMcAewMXADtFxPPF47sCHwH+pxhE\nPrbhPUcBR5OucnqO1K01mGnFuc1aypeWmg2TpG+R9pi9oeGxZSLiuRadbyfgPRFxcCuOb9bIxcDM\nzHxpqZmZuRiYmRkuBmZmhouBmZnhYmBmZrgYmJkZLgZmZgb8f4F+kVPHZjlNAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12634a8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt = runplt()\n",
    "plt.plot(X, y, 'k.')\n",
    "X2 = [[0], [10], [14], [25]]\n",
    "model = LinearRegression()\n",
    "model.fit(X, y)\n",
    "y2 = model.predict(X2)\n",
    "plt.plot(X, y, 'k.')\n",
    "plt.plot(X2, y2, 'g-')\n",
    "\n",
    "# 残差预测值\n",
    "yr = model.predict(X)\n",
    "for idx, x in enumerate(X):\n",
    "    plt.plot([x, x], [y[idx], yr[idx]], 'r-')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以通过残差之和最小化实现最佳拟合，也就是说模型预测的值与训练集的数据最接近就是最佳拟合。对模型的拟合度进行评估的函数称为残差平方和（residual sum of squares）成本函数。就是让所有训练数据与模型的残差的平方之和最小化，如下所示：\n",
    "\n",
    "$$SS_{res} = \\sum_{i=1}^n (y_i - f(x_i))^2$$\n",
    "\n",
    "其中，$y_i$是观测值，$f(x_i)$是预测值。\n",
    "\n",
    "残差平方和计算如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "残差平方和: 1.75\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "print('残差平方和: %.2f' % np.mean((model.predict(X) - y) ** 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有了成本函数，就要使其最小化获得参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###解一元线性回归的最小二乘法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过成本函数最小化获得参数，我们先求相关系数$\\beta$。按照频率论的观点，我们首先需要计算$x$的方差和$x$与$y$的协方差。\n",
    "\n",
    "方差是用来衡量样本分散程度的。如果样本全部相等，那么方差为0。方差越小，表示样本越集中，反正则样本越分散。方差计算公式如下：\n",
    "\n",
    "$$var(x) = \\frac {\\sum_{i=1}^n (x_i - \\bar x)^2} {n-1}$$\n",
    "\n",
    "其中，$\\bar x$是直径$x$的均值，$x_i$的训练集的第$i$个直径样本，$n$是样本数量。计算如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "23.2\n"
     ]
    }
   ],
   "source": [
    "# 如果是Python2，加from __future__ import division\n",
    "xbar = (6 + 8 + 10 + 14 + 18) / 5\n",
    "variance = ((6 - xbar)**2 + (8 - xbar)**2 + (10 - xbar)**2 + (14 - xbar)**2 + (18 - xbar)**2) / 4\n",
    "print(variance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Numpy里面有`var`方法可以直接计算方差，`ddof`参数是贝塞尔(无偏估计)校正系数（Bessel's correction），设置为1，可得样本方差无偏估计量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "23.2\n"
     ]
    }
   ],
   "source": [
    "print(np.var([6, 8, 10, 14, 18], ddof=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "协方差表示两个变量的总体的变化趋势。如果两个变量的变化趋势一致，也就是说如果其中一个大于自身的期望值，另外一个也大于自身的期望值，那么两个变量之间的协方差就是正值。 如果两个变量的变化趋势相反，即其中一个大于自身的期望值，另外一个却小于自身的期望值，那么两个变量之间的协方差就是负值。如果两个变量不相关，则协方差为0，变量线性无关不表示一定没有其他相关性。协方差公式如下：\n",
    "\n",
    "$$cov(x,y) = \\frac {\\sum_{i=1}^n (x_i - \\bar x)(y_i - \\bar y)} {n-1}$$\n",
    "\n",
    "其中，$\\bar x$是直径$x$的均值，$x_i$的训练集的第$i$个直径样本，$\\bar y$是价格$y$的均值，$y_i$的训练集的第$i$个价格样本，$n$是样本数量。计算如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22.65\n"
     ]
    }
   ],
   "source": [
    "ybar = (7 + 9 + 13 + 17.5 + 18) / 5\n",
    "cov = ((6 - xbar) * (7 - ybar) + (8 - xbar) * (9 - ybar) + (10 - xbar) * (13 - ybar) +\n",
    "(14 - xbar) * (17.5 - ybar) + (18 - xbar) * (18 - ybar)) / 4\n",
    "print(cov)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Numpy里面有`cov`方法可以直接计算方差。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22.65\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "print(np.cov([6, 8, 10, 14, 18], [7, 9, 13, 17.5, 18])[0][1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在有了方差和协方差，就可以计算相关系统$\\beta$了。\n",
    "\n",
    "$$\\beta = \\frac {cov(x,y)} {var(x)}$$\n",
    "\n",
    "$$\\beta = \\frac {22.65} {23.2} = 0.9762931034482758$$\n",
    "\n",
    "算出$\\beta$后，我们就可以计算$\\alpha$了：\n",
    "\n",
    "$$\\alpha = \\bar y - \\beta {\\bar x}$$\n",
    "\n",
    "将前面的数据带入公式就可以求出$\\alpha$了：\n",
    "\n",
    "$$\\alpha = 12.9 - 0.9762931034482758 \\times 11.2 = 1.9655172413793114$$\n",
    "\n",
    "这样就通过最小化成本函数求出模型参数了。把匹萨直径带入方程就可以求出对应的价格了，如11英寸直径价格\\$12.70，18英寸直径价格\\$19.54。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##模型评估"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "前面我们用学习算法对训练集进行估计，得出了模型的参数。如何评价模型在现实中的表现呢？现在让我们假设有另一组数据，作为测试集进行评估。\n",
    "\n",
    "|训练样本|直径（英寸）|价格（美元）|预测值（美元）|\n",
    "|::|::|::|::|\n",
    "|1|8|11|9.7759|\n",
    "|2|9|8.5|10.7522|\n",
    "|3|11|15|12.7048|\n",
    "|4|16|18|17.5863|\n",
    "|5|12|11|13.6811|\n",
    "\n",
    "有些度量方法可以用来评估预测效果，我们用R方（r-squared）评估匹萨价格预测的效果。R方也叫确定系数（coefficient of determination），表示模型对现实数据拟合的程度。计算R方的方法有几种。一元线性回归中R方等于皮尔逊积矩相关系数（Pearson product moment correlation coefficient或Pearson's r）的平方。\n",
    "\n",
    "这种方法计算的R方一定介于0～1之间的正数。其他计算方法，包括scikit-learn中的方法，不是用皮尔逊积矩相关系数的平方计算的，因此当模型拟合效果很差的时候R方会是负值。下面我们用scikit-learn方法来计算R方。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先，我们需要计算样本总体平方和，$\\bar y$是价格$y$的均值，$y_i$的训练集的第$i$个价格样本，$n$是样本数量。\n",
    "\n",
    "$$SS_{tot} = \\sum_{i=1}^n (y_i - \\bar y)^2$$\n",
    "\n",
    "$$SS_{tot} = (11 - 12.7)^2 + (8.5 - 12.7)^2 + \\cdots +(11 - 12.7)^2$$ = 56.8\n",
    "\n",
    "然后，我们计算残差平方和，和前面的一样：\n",
    "\n",
    "$$SS_{res} = \\sum_{i=1}^n (y_i - f(x_i))^2$$\n",
    "\n",
    "$$SS_{res} = (11 - 9.7759)^2 + (8.5 - 10.7522)^2 + \\cdots + (11 - 13.6811)^2 = 19.19821359$$\n",
    "\n",
    "最后用下面的公式计算R方：\n",
    "\n",
    "$$R^2 = 1 - \\frac {SS_{res}} {SS_{tot}}$$\n",
    "\n",
    "$$R^2 = 1 - \\frac {19.19821359} {56.8} = 0.6620032818661972$$\n",
    "\n",
    "R方是0.6620说明测试集里面过半数的价格都可以通过模型解释。现在，让我们用scikit-learn来验证一下。`LinearRegression`的`score`方法可以计算R方："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.66200528638545175"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 测试集\n",
    "X_test = [[8], [9], [11], [16], [12]]\n",
    "y_test = [[11], [8.5], [15], [18], [11]]\n",
    "model = LinearRegression()\n",
    "model.fit(X, y)\n",
    "model.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##多元线性回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看出匹萨价格预测的模型R方值并不显著。如何改进呢？\n",
    "\n",
    "回顾一下自己的生活经验，匹萨的价格其实还会受到其他因素的影响。比如，匹萨的价格还与上面的辅料有关。让我们再为模型增加一个解释变量。用一元线性回归已经无法解决了，我们可以用更具一般性的模型来表示，即多元线性回归。\n",
    "\n",
    "$$y = \\alpha + \\beta_1 x_1 + \\beta_2 x_2 + \\cdots + \\beta_n x_n$$\n",
    "\n",
    "写成矩阵形式如下：\n",
    "\n",
    "$$Y = X \\beta$$\n",
    "\n",
    "一元线性回归可以写成如下形式：\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "Y_1\\\\\n",
    "Y_2\\\\\n",
    "\\vdots\\\\\n",
    "Y_n\\\\\n",
    "\\end{bmatrix}\n",
    "=\n",
    "\\begin{bmatrix}\n",
    "\\alpha + \\beta X_1\\\\\n",
    "\\alpha + \\beta X_2\\\\\n",
    "\\vdots\\\\\n",
    "\\alpha + \\beta X_n\\\\\n",
    "\\end{bmatrix}\n",
    "=\n",
    "\\begin{bmatrix}\n",
    "1 & X_1\\\\\n",
    "1 & X_2\\\\\n",
    "\\vdots & \\vdots\\\\\n",
    "1 & X_n\\\\\n",
    "\\end{bmatrix}\n",
    "\\times\n",
    "\\begin{bmatrix}\n",
    "\\alpha\\\\\n",
    "\\beta\\\\\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "其中，$Y$是训练集的响应变量列向量，$\\beta$是模型参数列向量。$X$称为设计矩阵，是$m \\times n$维训练集的解释变量矩阵。$m$是训练集样本数量，$n$是解释变量个数。增加辅料的匹萨价格预测模型训练集如下表所示：\n",
    "\n",
    "|训练样本|直径（英寸）|辅料种类|价格（美元）|\n",
    "|::|::|::|::|\n",
    "|1|8|11|9.7759|\n",
    "|2|9|8.5|10.7522|\n",
    "|3|11|15|12.7048|\n",
    "|4|16|18|17.5863|\n",
    "|5|12|11|13.6811|\n",
    "\n",
    "我们同时要升级测试集数据：\n",
    "\n",
    "|测试样本|直径（英寸）|辅料种类|价格（美元）|\n",
    "|::|::|::|::|\n",
    "|1|8|2|11|\n",
    "|2|9|0|8.5|\n",
    "|3|11|2|15|\n",
    "|4|16|2|18|\n",
    "|5|12|0|11|\n",
    "\n",
    "我们的学习算法评估三个参数的值：两个相关因子和一个截距。$\\beta$的求解方法可以通过矩阵运算来实现。\n",
    "\n",
    "$$Y = X \\beta$$\n",
    "\n",
    "矩阵没有除法运算（详见线性代数相关内容），所以用矩阵的转置运算和逆运算来实现：\n",
    "\n",
    "$$\\beta = (X^TX)^{-1}X^TY $$\n",
    "\n",
    "通过Numpy的矩阵操作就可以完成："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.1875    ]\n",
      " [ 1.01041667]\n",
      " [ 0.39583333]]\n"
     ]
    }
   ],
   "source": [
    "from numpy.linalg import inv\n",
    "from numpy import dot, transpose\n",
    "X = [[1, 6, 2], [1, 8, 1], [1, 10, 0], [1, 14, 2], [1, 18, 0]]\n",
    "y = [[7], [9], [13], [17.5], [18]]\n",
    "print(dot(inv(dot(transpose(X), X)), dot(transpose(X), y)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Numpy也提供了最小二乘法函数来实现这一过程："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.1875    ]\n",
      " [ 1.01041667]\n",
      " [ 0.39583333]]\n"
     ]
    }
   ],
   "source": [
    "from numpy.linalg import lstsq\n",
    "print(lstsq(X, y)[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "有了参数，我们就来更新价格预测模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted: [ 10.06250019], Target: [11]\n",
      "Predicted: [ 10.28125019], Target: [8.5]\n",
      "Predicted: [ 13.09375019], Target: [15]\n",
      "Predicted: [ 18.14583353], Target: [18]\n",
      "Predicted: [ 13.31250019], Target: [11]\n",
      "R-squared: 0.77\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "X = [[6, 2], [8, 1], [10, 0], [14, 2], [18, 0]]\n",
    "y = [[7], [9], [13], [17.5], [18]]\n",
    "model = LinearRegression()\n",
    "model.fit(X, y)\n",
    "X_test = [[8, 2], [9, 0], [11, 2], [16, 2], [12, 0]]\n",
    "y_test = [[11], [8.5], [15], [18], [11]]\n",
    "predictions = model.predict(X_test)\n",
    "for i, prediction in enumerate(predictions):\n",
    "    print('Predicted: %s, Target: %s' % (prediction, y_test[i]))\n",
    "print('R-squared: %.2f' % model.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "增加解释变量让模型拟合效果更好了。后面我们会论述一个问题：为什么只用一个测试集评估一个模型的效果是不准确的，如何通过将测试集数据分块的方法来测试，让模型的测试效果更可靠。不过现在我们可以认为，匹萨价格预测问题，多元回归确实比一元回归效果更好。假如解释变量和响应变量的关系不是线性的呢？下面我们来研究一个特别的多元线性回归的情况，可以用来构建非线性关系模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##多项式回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上例中，我们假设解释变量和响应变量的关系是线性的。真实情况未必如此。下面我们用多项式回归，一种特殊的多元线性回归方法，增加了指数项（$x$的次数大于1）。现实世界中的曲线关系都是通过增加多项式实现的，其实现方式和多元线性回归类似。本例还用一个解释变量，匹萨直径。让我们用下面的数据对两种模型做个比较：\n",
    "\n",
    "|训练样本|直径（英寸）|价格（美元）|\n",
    "|::|::|::|\n",
    "|1|6|7|\n",
    "|2|8|9|\n",
    "|3|10|13|\n",
    "|4|14|17.5|\n",
    "|5|18|18|\n",
    "\n",
    "|测试样本|直径（英寸）|价格（美元）|\n",
    "|::|::|::|\n",
    "|1|6|7|\n",
    "|2|8|9|\n",
    "|3|10|13|\n",
    "|4|14|17.5|\n",
    "\n",
    "二次回归（Quadratic Regression），即回归方程有个二次项，公式如下：\n",
    "\n",
    "$$y = \\alpha + \\beta_1 x + \\beta_2 x^2$$\n",
    "\n",
    "我们只用一个解释变量，但是模型有三项，通过第三项（二次项）来实现曲线关系。`PolynomialFeatures`转换器可以用来解决这个问题。代码如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEaCAYAAADzDTuZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XXcW+X5x/HPl1KsSHGH4l6KWweF4TB0uBcpwx0KGzLY\nhttwl2HDhg0GY8UZXlzGfnSsgyGDDhhSoN/fH/d5SHh4PHJOkuv9euXV5CQ553puQq7cLtuEEEJo\nbRPlHUAIIYT8RTIIIYQQySCEEEIkgxBCCEQyCCGEQCSDEEIIRDII4Xsk1e3/CUnHSZq8D++buBbx\nhNYWH6pQCJKmAh4Gupr4shjwaHZ/KeC5Du6/bntPSZMBN9jeODv/XsDuwBq2Py677vZAP9tXZl/M\nT0g6Afg2e8kjtt/LXvuM7WW6+BvmAWYrO7Q58Clwb9mx/9h+TdJ2wEzA4pJOLnt+emBW2zNKmhdY\nHhgAjAPmBG4BjpZ0lu0Xy659AnAHsB6wYXbdNrMCh9u+rbPYQ4hkEIqiH/Cx7dUlrQ6sbXukpKds\nL9f+xZIOs31ydv8c2/u0e8mmwF+y59cFFga2Aa6StJ3tTySdB6yRXqLZSF+6LwGPANMA1wHrZOfo\nBywg6fFO4n8euBCYoy3E7Bz9y461xb4usBewYZaYVs+OT0/6Qt8ie+lbpITyMTAVMJnttyWtBIyS\nNJiUJKYFViElksmAEbafKbverp3EHMJ3IhmEIpoU+Lr9QUkLAPuTag/Plz01naQ5bI8tO7YX8AtJ\nRwGzA/vYniDpGOAOSZfY3kvSL4F3svMtnb13C9Iv7IPaagXZOR6yvWFXgUtaGtgN+IyUXExKRFMA\n19m+QdIpwEbABZKOtv26pJWBa4BrbT+QnW4ksBap5jMW+EDSOOD17Pz7AFcDr2Z/w6vA/MC5kspr\nBtMDR3UVdwiRDEIRTc33mznazE76hTwWWFDSOcCKwEfAtZKmBk4FvgHmBaYkfUG+CyxB+sLfAWir\nKQBsTPrCPgU4D9gO+A3wL2C8JDmt2TIjsKKkpzqJeRXb47NrjgL+SqodtBlMqilg+1BJ6wCfAAMk\nXQNMTmre2U/SnaSkdyLp/9H+2b8nAkcA79m+TdJ+wB+yWs7/AU+Qagl7235G0uHAPbbLE2cIHYpk\nEIpoYeCF8gNZ08oSwCvAj0lf2uOAS4ERwLHAKNv3SDoduJzU7PRo1p7+RnaqIcBXtp+TtAbwX+AS\nYAHgfuBu0q/ruYFDgAOBzbNmlxk6CjbrSxhfduhbSn0OkJLCt9m/SJoTuALYHhgOnGO7rflphKRh\nwK6kpPYE8CHwd2A14CDgrKwTeYosEcwFLApsQkp+G0j6HzAI2CKrJbxue8+O4g8BIhmEgpE0E6m9\n/9Ts0NdZW/oQUtv5QFICGE5qQjFwe/bapSX9CDgYOLrstNOSvvTbGw7cQPqSPp/UobtRdmuzeW//\nBOAnpBpLuemAm7P7vwMeJ33ZjwJ+LQlSR++72WsOA7bN/o55gJeBT20fKekV4LTsHABHkhLV/qQm\nroNsj82axO4s7z8IoTORDEJRmDTU+Wpgf9ufZ8fPAe4k/Tq+gtRU9B7pC3uC7bU7Oln25do2MmlJ\nSjWDcoeSOpAntv2epEVs/6jsHJcDk0u6hTSSp6OYj+P7zUEmfek/2u74UqS+EICfkvo0sH1z9nok\n3W17vbLrv0YaYTVPdv3Ds6fOA0YDC2Xn2FPS8cD/steNkLQWqRlqaFZz2MP2gx38DSEAkQxCcQiY\nAKxne0LbQdvXAtcCZM0nk5C+wG+hNOqm03Nm73nH9lftn7T9bpY0umR7sy4usBDw73Z/xwBSbUSk\n5CBSX8f47Hwf9PC6n0oaSWq+mhFYO5sHcTipT+JASXtnfRoTkfokviTVOPZuqxFIOrjbi4WWV/Nk\nkLWPXkWqghu4yPbZko4ljbr4IHvpSNv31DqeUFjfkJqAnunii3IyUl/AGaQhmLv14LyHA3uXPR7P\nD+cyfPdY0sNlx+cHjml/QknHAVsBX5FGCe2THW8bqjqeNNSz3JSkzuLBtrcqO9eostcMKHv8Fqn/\nYlvgsey2P7Ag8Jjt7SSdD9wk6R+kJjMBZ5OGqpaPKJoT6KzjOwQAVOvNbSTNAsxie7SkKYFnSB1d\nW5LaQE+vaQAhNChJywGv2f607Njytp8sezyX7bez+20jn0LotZrXDGz/m6wabfszSa+S2n3h+22q\nIYQytn/wa748EWSP3y67H4kg9Fld1yaSNIjUkfbX7NC+kp6XdKmkgfWMJYQQQkk9F+WaEriJNFLk\nM9JQvnlIQwbfJQ2VCyGEkIOa9xkASOpPGh54t+0zO3h+EHCH7SXaHY9qbwgh9IHtXjXD12M0kUiT\nhF4pTwSSZrXdNsFmU+DFjt7f2z+oWUk61vaxecdRBFEWJVEWJY1UFhL9gANI60/9Cjjb/t6s9QrP\n3/sf0vWYZ7AKadr9C5Lalhk+EthG0hDSsL63SEsKhM4NyjuAAhmUdwAFMijvAApkUN4B9ITE4sBl\npMUGV7D5e84hAfUZTfQIHfdN3F3ra4cQQlFITEKqCexD+kF8id3l/h11FTOQG8cVeQdQIFfkHUCB\nXJF3AAVyRd4BdEZiOVJtYAywlM3Yrt9Rf3XpQO6rbA5N9BmEEBqSxBSk9at2IK2Ae309agN9+e6M\nPZAbRLbGTiDKolyURUnRykJiNdIeGnMAS9hcV6RmofaimSiEEKpIYmrgJNJmRXvb3y2xXmjRTBRC\nCFUisT5wAfAn4FCbcfnE0fvvzqgZhBBChSRmAM4EVgJ2tvlLziH1WvQZNIiitYfmKcqiJMqiJI+y\nkJDElqRJs+8DgxsxEUDUDEIIoU8kZgPOJe0xsan93QKcDSn6DEIIoRckBOwCnEjqH/iVzQ920stT\n9BmEEEINScwDXETa1nQtm+dzDqlqos+gQUTbcEmURUmURUkty0Kin8R+pO1D7wNWbKZEAFEzCCGE\nLkksAlwCfAusbPNGziHVRPQZhBBCByT6A4eRlpo+GrjQZkK+UfVM9BmEEEIVSCxN2ofl38AyNm93\n85aGF30GDSLahkuiLEqiLEqqURYSk0n8hrTE/hnA+q2QCCBqBiGEAIDEKqTawIukyWPv5RxSXUWf\nQQihpUlMBfwa2AzY1+aWnEOqWCxhHUIIvSCxNqkmMABYvBkSQV9FMmgQ0TZcEmVREmVR0puykJhW\n4nLSBLI9bIbbfFyz4BpAJIMQQkuR2Ax4CfiUVBu4N+eQCiH6DEIILUFiFuAcYAlgV5tHcg6pZqLP\nIIQQ2smWmd6RtAXlG8CSzZwI+iqSQYOItuGSKIuSKIuSjspCYi7gLtJm9OvZHGnzZb1jawSRDEII\nTUdiIom9gGeAR4DlbZ7NOaxCiz6DEEJTkViQtLBcP1LfwGs5h1R30WcQQmhZEhNLHAY8BtwMrNqK\niaCvIhk0iGgbLomyKImySCQGw50vA2sBy9mcZfNt3nE1kkgGIYSGJTGpxPHAn+HlPwBr27yVd1yN\nKPoMQggNSWJF0sJyfwP2snkn55AKI/YzCCE0PYkBwPHANsD+wI02xf1V2yCimahBRNtwSZRFSauV\nhcQawAvAjKSlJH7flgharSyqLWoGIYTCkxgInAKsA+xp88ecQ2o60WcQQig0iY2A84A7gcNsPsk5\npMKLPoMQQtOQmBE4G1gW2M7mwZxDamrRZ9Agoj20JMqipBnLIltYblvSpjNjSQvLdZsImrEs6ilq\nBiGEwpCYAzgfmBv4ic1TOYfUMmpeM5A0p6RRkl6W9JKk/bLj00m6T9Ibku6VNLDWsTQy2w/kHUNR\nRFmUNEtZZAvL7QE8BzwNLNvbRNAsZZGXmncgS5oFmMX2aElTklYR3ATYBfjQ9smSDgemtX1Eu/dG\nB3IITU5iPtLCclOQFpZ7KeeQGl4hF6qz/W/bo7P7nwGvArMDGwFXZi+7kpQgQieiPbQkyqKkkctC\nop/EQcATpJFCK1eSCBq5LIqgrn0GkgYBS5H+489s+73sqfeAmesZSwghPxKLAZcBnwMr2ryZc0gt\nr27JIGsiuhnY3/anUqkGY9uSOmyvknQFMCZ7OA4Y3dY22PZLoBUe236gSPHE4+I8blOUeLp+PHBi\n+HhlYF845Uo46i57/JvVOH/bsWL9vfV5nN3fOSuGMfRBXSadSepPqgbebfvM7NhrwDDb/5Y0KzDK\n9sLt3hd9BiE0CYnlSAvLvU2aRTw255CaViH7DJSqAJcCr7QlgsztwE7Z/Z2AP9Q6lkYW7aElURYl\njVAWEpNLnAzcAZxIGjJa9UTQCGVRZPVoJloF2B54QdJz2bGRpA/F7yXtSqrWbFmHWEIIdSSxGmmk\n0DPAYJv3cw4pdCLWJgohVJ3E1MBJwE9Iew3cnnNILaWQzUQhhNYisR5pKYn+pGWmIxE0gEgGDSLa\nQ0uiLEqKVBYS00tcBZwLDLfZzWZc/a5fnLJoRJEMQggVyRaW2wJ4CfgIWMLm/pzDCr0UfQYhhD6T\nmJW018DCpKUkHss5pED0GYQQ6iSrDQwHnifVCIZEImhskQwaRLSHlkRZlORRFhKDgD8BewFr2fzC\n5qt6x9FefC4qE8kghNAj2TLT+5GWmL6ftKbQ8zmHFaok+gxCCN2SWJi0ksAEUt/AGzmHFLoQfQYh\nhKqS6C9xJPAIcC2wWiSC5hTJoEFEe2hJlEVJLctCYingSWBVYBmbc20m1Op6lYrPRWViD+QQwvdI\nTAYcDewGHApcZeOyF/QDZgBmAqYFpgEGAlMCkwGTA5MC/YC2pooJwHjgq+zfT4FPsts44IPs9hlF\nbrtuYtFnEEL4jsQqk/LlpUMY/Y+L2f2aJXhpRtLm9HNlt9mB6Uhf4O+TJpn9N3v8GfAF8CXpS/9b\nwNmtHzBJdpuMlDimAqYmJZQZScllIuAd4J/AWOAfwJvA37N/341k0b2+fHdGMgihVaXl5ecGlvmM\nAcu8zGJbzsCHgwYxZkI/JvwDeIP0BTyG9KX8NvAv4APsb3p4iYuABUk7mm1ru+vlKaQBpIQzBzBn\nFt98wPzAAqRk8gppbsOLwLPAaOz/9eIvb3qRDJpY+Q5OrS7KoqRXZZF2G1wRGAqsACwLfPMvZhtz\nPVsv8i6zPrcu9xy5Jvc/jf11leJ7AFgte/h721tVeMIZgMWy22Bg6ez+P66FMdvCLcBjwGvYhe3f\nqLW+fHdGn0EIzSr9yv4R8GNgddKSEc+RRgZdcCU7vrEzVx4KrAFsZfMnOKTaUXye/fsUMKLis9kf\nAg9mtyTtpLjYu7AjKfGMBAYiPQiMym4vR/NS16JmEEKzSM0+iwLrZ7flSJvK3A/8BXgK+6v0UjYF\nzgFuBUbafFqjkAYCFwIjum0iqu6FZweGkZLgGqQO7XuAu4E/U89YchDNRCG0mjSyZxVgM2BjUgfs\nXcAfgVHt29IlZgZ+CwwhTR57uL4B50RaAFgXWI/UTPYEaavd27Cbbi/mmHTWxGIMdUkzloWkiyQ9\nIOmP2a/prl48EdIwpAvvgw+Bs4D/ABsBg7D3wr6zPBFkC8vtALxAGpmzZLMlgi4/F/bfsH+LvT4w\nK3A+qd/keaTHkQ7IahMtK/oMQiiGBSl1tF4I/LCjVVqS1C6+NWlM/nWnw15r2dd1dWKJuYALgNmA\n9W2eqWLcjSclyVuAW7L+hh+TyvRopBeAq4CbsD/JMcq6i2aiEApA0h9JTRhPAWt/176eRs9sC+xC\nGt9/FXAt9qvdn5OJgD2B40i1h5NsqjJKqClJk5L6WnYk9TXcBVxGam5rqJFJ0WcQQoP6XkdrmsQ1\nlPRFvj7pS+lyevGlJLEAcAlpXP6uNq/UJPBmlZLwNsDupBnVFwNXYL+fa1w9FH0GTawZ28n7qhnL\nwvY4w3CnL6CXgItItYT5sLfHvr+jRNC+LCQmljgMeJzUFDK0VRJBVT8X9ofYvwWWBLYnDct9Hel3\nSMtX7ToFEskghLxJcyGdTJrluzawD7Ao9pnYH/X8NAwmJYG1geVtzrL5tiYxtwrb2E9gDyfNhB4N\n3ID0BNLWSE3T7xrNRCHkRRpMWghufeBK4Bzs/+v9aZgUOAr4GXAEcNn3FpYL1ZWG824IHERaLuMM\n4FLsz3KNq0w0E4XQCKRVSB3G95CahObDPqiPiWAF0vo8S5KGi14aiaDG7G+xb8NejTTqayjwFtIx\nSNPmHF2fRTJoEM3YTt5XDVsW0mpI9wNXkyY8zYt9Ul9mw0oMkDgd7vsj8EtgE5t3qhxxQ8nlc5Ga\nkLYgTfybG3gT6ddIM9Y9lgpFMgih1qShpAXbLiElgoWwL8L+sm+nYw3S5LGZ4LDhNjdEbSBn9htZ\nv8IypCW5X0M6oZFqCtFnEEKtSEsDJ5DWCzoOuLqnSz93fDqmAU4hzUfY0+auqsQZqk8aBPwc2IQ0\nx+OMevYpRJ9BCEUgzYt0PXAnaY7AQtiXV5gINiT1LxhYPBJBwdljsHcjLRm+CPAG0s+yGc+FFMmg\nQTRsO3kNFLYspOmRziDND3gZWAD73LaVQvt2SmaUuBY4E9jRZoTNf0vPF7QsclDIsrDfxN6WNPpo\nM+AlpE2yFWYLJZJBCJWS+iPtB7xKWip5UezjK9l9K1tYbhvSbl7vAoNtRlUn4FB39rPYawH7kpoO\n78/WmiqM6DMIoRLSOqRx5mOBA7FfrvyUzE5aVXNe0lIST1R6zlAgaaLa7sCxwG3AUdgfVPcS0WcQ\nQn1IcyPdCpxLmui1TqWJIKsN7E6a5fossHQkgiZkf4N9PrAQaSe4l5H2zCaz5SaSQYMoZHtoTnIt\nC2kSpCNIO4g9CyyOfXulWypKzAv8GdgDWMPmWJvx3b8vPhdtGq4s7HHYBwBrAtsBT+a57lEkgxB6\nShpK+tU+FFg+6xfo01yB0inpJ3Eg8CRpS8aVbF6sPNjQMOwXgFVJgwRuQzobaep6h1GXPgNJlwEb\nAO/bXiI7diywG2mTDoCRtu9p977oMwj5S8tLn0gaEbIfcGs1NleXWAy4FPgS2M3mzUrPGRqcNB1p\nLsnawH7Yt/btNMXtM7ictP9oOQOn214qu93TwftCyJe0MWl8P6QmoVuq0CQ0icQvgAeAK0jNQpEI\nAtgfYe9KWjb7RKQbkWaux6W7TQaS+knaMdub9dXs9pKkyyWt2JOL2H4Y+Lij0/c24FbVcO2hNVSX\nskhzBq4BTgW2xd6zL2sI/fC0LEuah7ASqYP4Aps+76IVn4uSpioL+0FgCPB/wAtI29Z6bkKXyUAp\nI90FzAHsbXsR24uQVki8ANhD0nkVXH9fSc9LurTbTcBDqBdpU9L4/veAJbEfqvyUTC5xEun/p1OA\nDWz+Wel5QxOzv8A+nNQ8eSRwK9JMtbpcl30GkuYFvrD9bhevWc0pi3V9obRWxx1lfQYzUeovOB6Y\n1al6VP6e6DMI9SNNA5xNWoFyZ+xHqnNaViUtUvccsK9NQ2ydGAok7c98LGkv7L2wb+n65b3/7uxy\nlx73YH31niSCTt733f8Qki4B7ujodZKuAMZkD8cBo20/kD03LDtXPI7HlT2Whv0Jrv8AntwehmB/\nVvn5518fjtkDdlgO2Bs0DlgU0me/UH9/PC72Y/srSX8aAWMvgJOQNlkIbngD/mf7gey1O5OMoQ/q\nNgO5g5rBrG01DkkHAss5reFR/p6oGWQkDWv7kLS6qpaFNAmpZroDsDt2VRaAk1iPtMH9fcAhdod9\nZlW4Tnwu2rRMWUgDgNOAdYDtsB/74UuqXDOoFknXAasBM0j6J3AMMEzSENKooreAEfWIJYTvSAsA\n15HW/lmyGksCSExPWp7iR8Bwmz9Xes4QvietebUn0iakfoTzgRMqWRUXelEzkHSu7b2z+ycDywGX\n2L6mkgC6uWbUDEJtSDuRRgodB5xbheGiAn5K6nP4PXCUTWH2xA1NSpqNtH/2ZKRRb/9Mh3v/3dld\nB/LvbG+f3X/c9kplz80D3GN7oT78CT0LLpJBqLZUxT4XWAHYKpv9WeEpmTU756Kk2sAPqu0h1Iw0\nEXAYcACwK/ZdtZh0Nl9nT9h+C/iwNxcLfddUY6gr1OeykBYjjfGfCFiu0kSQLSy3C/A8afnqIfVO\nBPG5KGnZsrAnYJ9Iqpmej3RqX07Tmz6DySTd2O69A/py0RDqTtoBOJ30C+qKKjQLDQIuAmYA1rYZ\nXXGMIVTCfgRpKVKzUa9110z0XdOQpL8CO8F3s4YNvG37i75cuEfBRTNRqFQan30GaWXIzbErWgRO\nYiJgb9IgiNOAU22+rjjOEKpFmkjwbS1HE9n2670MK4T8SHMCNwH/IjUL/bebd3RzOhYiLSwHMNTm\ntQojDKH67An0YeWK3ixUN4XSmkSPSTozmzcQ6qRl20M70KOykFYlLQt9M6lG0OdEINFfYiTwKHA9\nsGpREkF8LkqiLCrTm5rB57ZXypaR2BC4V9LWtp+tUWwh9J4kYC/gF8D22BWN85cYAlxGGiyxrN23\n2Z0hFF13fQblcwvaDy0dDFwLLG272x2Z+hRc9BmE3kj9A+cBywMb04PlVDo/FZOREsruwOHAFTbF\n3TA8hDJVH1ralggyD7Z77gXS/3gz9OaCIdREqrHeDwwEVqowEaxMWlRuYWCwzeWRCEKz68l+BodI\nmhg4WtI52bHjs6fHArPWML6QifbQkh+UhbQE8AQwCtgCu08zfyWmlDiL1On8c5vNbf5dYbg1FZ+L\nkiiLyvSkz2AD0gYL3wKLZceGZv+uDtxeg7hC6BlpA9JOegdgX9v307AmcDHwELC4zUdVijCEhtBp\nMpC0MDB/20P4/q5kSh11y5Mm8YQaa4nVGHvou7KQ9gVGkvoHHu/LuSQGkuYLrAnsaXN3lcKsi/hc\nlERZVKarZqJ/AgcD/UkTzNq3mR4AXGk7JtyE+pL6IZ0B/AxYpYJEsAnwMmlD+sUbLRGEUE2dJgOn\nZVLXA6bJ/l0fmFHS1sBSwPS2L6pLlCHaQzPTSpeeCh89AzucCOuR1sjqFYmZJW4ATga2ttnb5tPq\nR1t78bkoibKoTHejib4EJpB+OX2V3f+c1Lw0h9JqeSHUhzTjQ7DFhzD1SjD9yPRl3ou3I4ntgRdI\nu0EtafNwLUINodF0Os9A0uSk2ZbTAL8ldSDvb3t1SaNIm4LMbfuomgUX8wxCm7Rk+p8ugolGpNV0\nnwLWtj2uZ29nTtLOY7MDu9o8XbtgQ8hXtecZLE5a4Mt00GeQNRGtqrSJeAi1Iy0NPAKcOQKWJW0e\n06NEIDGRxM+AZ4HHgeUiEYTwQ131GTxV1js/Ibu1dyewVg3iCu20bHuotDpwD7Av9nlZAji/h4lg\nAdLcg52A1WyOt6nJbPm8tOznogNRFpXpSZv/Dbb/YPsOoG3533uyfx8APqlFYCEgbQbcQJpIdkvP\n38bEEoeSagK3AqvYvFKjKENoCj3eAzkP0WfQwqRdgeOBDenFYogSg0nLTH8C7G7T52UpQmhUtdj2\nMoT6kw4mLRI3rKeJQGJSieNI6xNdCKwZiSCEnotk0CBaoj1UEtJxpJVCh2K/0cnLhn3/MSsAz5Dm\nvwyxuaRVFpZric9FD0VZVKbLZCBpPkmzdfOaoV09H0KPpOVNTgM2AVbFHtv9W5hC4jTgNuAEYGOb\nf9U20BCaU3f7GcwCXAE8DFzrstmekpYF9gS+sb1nTYKLPoPWkCYvng8MIc0q7naROInVgUtIq5Xu\nb/NBbYMMoXH05buz2w7kbPnqrbPbvKQF6yaQxm1faPuRvoXbg+AiGTQ/qR+pw3ceUmdxl8tCSExD\nmnm8PrCXzR21DzKExlKTZJCnSAYlkoY13aqM6YfGFaQ9MTYirYfVxcvZEDgfrhkN221vU9EG982g\nKT8XfRRlUdKX787e7IEcQvVI/YGrgemAn2B/3vlLmRE4C1gB2BG2t71dyyeCEKopagah/lIiuAaY\nEtiMtCBiBy9DwFbAmdnrj7bpsvYQQoiaQWgEqWnod3SfCGYn7bE9P2mU0BP1CzKE1tPd0NKZJc2c\n3ZekmyXNJGnu7DZXfcIMTTGGOiWCq0kr4XaYCLJlpncDRme3pdsngqYoiyqJsiiJsqhMdzWDocAv\nJB0EjAf+A9wMvJk9vy6p8y+ErqVRQ1eS+gg27iQRzEvah3ga4Mc2L9Q3yBBaV3fzDDYHXiEtZT0Q\n2BG41PaPsucfbrtfk+Ciz6A5pHkElwJzkjqLv/j+0/QD9gV+DpwEnGHzTd3jDKFJ1KLPoB/wX+Bu\n4BDgH32MLbSqNLP4HFLb/7odJIJFSYliPLCSzd/qH2QIodM+g2xLy58DpwIzAiOB/eoUV2inIdtD\nS0tMLANsUD6PQKK/xM+BB4GrgNV7mggasixqJMqiJMqiMp3WDGxPkLQCcBTwLmlo3/3Ai9m2l0BU\n5UOXfgmsDqyB/d2+FxJLA5eRPlfL2LydU3whhExPlqMYSdrEZl5gAdIuU+/VPrToM2ho0iHArqRF\n5z5Ih5gcOAbYBTgYuKZVVhcNoZ6q3mcgaSdgZ+Bj0qJ0pwPrpto/ALZ9VQ8CuwzYAHjf9hLZselI\nu1jNDYwBtuzp5uah4KTdgb2BH5UlgqGkvoHRwGCbuvygCCH0THf7GXwLGJiNtEDdbqTJQt9kt297\neJ3LScNQyx0B3Gd7QVLz0xE9PFdLapj2UGkr4FhgbeyxElNJnENK/EfYbFVpImiYsqiDKIuSKIvK\ndFkzsP07SSsDb5BqCNMCw4FDbb/b04vYfljSoHaHNwJWy+5fSWqKioTQyKR1gLOBtbD/JrEOadex\nvwCL23yca3whhE71pM9gauAO0i/78dnhtjfZPVzcKEsGd5Q1E31se9rsvoCP2h6XvSf6DBpFGmxw\nJ7CJ8KukJsXVgBE29+YaWwgtphZ9BtOShv0dCPyatPlIufHAOr25YEdsW1J0JDYqaRHSbmM7C88C\n3JjdlrD5LNfYQgg90t2ks78A29h+jbSZTTW9J2kW2/+WNCvwfkcvknQFqYMZYBwwum3N8rY2wlZ4\nXN4eWoR42h4PgxlHwWlvMt+vFuA3h8Hd88B6W9g8KmmYVP3rty+TIpVHDo+H2D6zQPHk+fgAWvv7\nYWeSMfRAb0ghAAASuUlEQVRBd8tR7AUsDBwALAFMRZqR/Ibtr3p1oR82E50M/Mf2SZKOAAbaPqLd\ne6KZKKMibtwhTWt45HY2emETbluDtA3l8TYdrkRavcsWsCxyEmVREmVR0pfvzp70GaxL6ux9mjTX\nYFpgSeAd4Ajb/9eDwK4jtR/PALwHHE1qVvg9MBedDC2NZFBg0mSfM/mDN/HTmXbiqnHAcJvn8g4r\nhFCjZJCd+CjgVdu3lB0bBhxue73eBtrj4CIZFNJF2mPilXnsyTdYcNGtuOGX39D/FJuv844rhJD0\n5buzu/0MbpO0P/AoMF7SKtltZeBr4IS+hxt6oyhjqCfVVwtNxpf/+ILJ5xvJb5b72v1/Xe9EUJSy\nKIIoi5Ioi8p014G8FbApcDypiehy+MH/+I/WIK5QMBITAwcfylm/WJd7PvmYaed53Qt9lHdcIYTq\n6PEeyJK2BpYHDu7p3IJKRTNRMUgsCVy2Gxf3O5+fzTwx366AHYvLhVBQNeszyEskg3xJTEpaxnzE\nXpx74TnsM0JpT4IfDDOWdBGwIPA5sG37wQAhhPqpep9BKI56t4dKrAQ8Byx2Mbv95Fz22VWwS0eJ\nILMgacTYeqQlKGoYW7QNt4myKImyqEx3fQahxUgMIA0M2BrYbxzT3DcNnzwOnIB9Vxdv/Tz79ylg\nRI3DDCFUWTQThe9I/Ji0If0jwIFGn5C2PH0J+4Cu36uBpBrBiGgiCiFf0WcQ+kRiIGl707WAPW3u\nRhJwATAHsBF2T5crDyHkLPoMmlit2kMlNgJeIi06uITN3dlTBwIrAVsXLRFE23BJlEVJlEVlos+g\nRUnMRNp7YGlgW5uHyp7cEDgEWAn703wiDCHUUzQTtRgJAdsCp5GWJz/G5ouyFywGjAI2xn48lyBD\nCBXpy3dn1AxaiMQcpH6AuYANbZ5u94LpgduBQyIRhNBaos+gQVTSHioxkcQI0ryBJ4FlO0gE/YGb\ngJuwr6og1JqLtuGSKIuSKIvKRM2gyUnMTxouOjkwzOblTl56FvA/4Mh6xRZCKI7oM2hSEv1ImxKN\nBH4FnG3T8aggaXfgIGAF7E/qFmQIoSaizyAAILE4cCnpl/4KNn/v4sWrkJLF0EgEIbSu6DNoED1p\nD5WYROIY0migS4Afd5MIZiftNrcz9htVCrXmom24JMqiJMqiMlEzaBISywGXkbYQXcpmbDdvmAy4\nBTgH+481DzCEUGjRZ9DgJKYAfglsT5o1fL1Ntxtbk2oOUwNbUuQPQQih16LPoMVIrEb6Un+atJTE\nBz186+7AiqQO40gEIYToM2gU5e2hElNLnA/8DjjIZpseJwJpRdIS1Ztif1aLWGst2oZLoixKoiwq\nE8mgwUisT1pYbmJSbeCOzl+riyQ9IOmPkgYizQzcCOzWSB3GIYTaiz6DBiExA3AmaSXR3W3+0v17\n9ABp9zH6w43jYSbgIeyjaxlrCCFfsYR1E5KQxJbAi8D7wOCeJILMd7uPfQjvAF8Bx9UgzBBCg4tk\nUGASs5GGfx4Dexxvc5DN/3pxim2B34+Bs6aGTYHtirY3QV9E23BJlEVJlEVlIhkUUFYb2BUYTaoR\nLA0Xv9Lb89geZzhqbjgD2AL7w2rHGkJoDtFnUDAS8wAXAdMCu9o8X8HJJgf+ClyEfW51IgwhFF30\nGTQwiX4S+wNPAfcCK1aUCJLfAi8D51UaXwihuUUyKACJRYCHgc2BlW1Osfnm+6/pZXuotCMwFBjR\nbBPLom24JMqiJMqiMpEMciTRX+Io4CHgatJ+A5WP/5cWJW1ruUXsYRxC6InoM8iJxNKkheXeBUbY\nvF2lEw8g7WZ2KvblVTlnCKGh9OW7M5JBnUlMBhwDDAcOBa7udmG53l3gCgDsnat2zhBCQ4kO5IKT\nGAo8D8xPmjx2VU8TQY/aQ1M/wfLA3hWEWXjRNlwSZVESZVGZWLW0DiSmAn4NbAbsa3NLDS6yEKmf\nYA3s3kxMCyGEaCaqNYm1SfMG/gIcbPNxDS7SNp/gXOyLqn7+EEJDiT6DApGYjvRLfXVgD5t7a3ix\n84Dpga2bbRhpCKH3GrLPQNIYSS9Iek7Sk3nHUw0Sm5GWkfgUWLwaiaDT9lBpM2AdYI9WSQTRNlwS\nZVESZVGZIvQZGBhm+6O8A6mUxCykWb9LAFvaPFrjC84JnA9shP3fml4rhNDUcq8ZZBqyKahNtrDc\njqSRQn8DhlQ7Edh+oN1F+5F2OjsD+4lqXqvoflAWLSzKoiTKojJFqRn8WdK3wIW2L847oN6QmAu4\nEJgVWM/m2Tpd+kjgG+DkOl0vhNDEipAMVrH9rqQZgfskvWb74bYnlSZRjckejgNGt/0CaGsjzOOx\nxERw+mkweDiseQpwEmgViWG1uV6pPdTwNbD3UNj7UVjVkHt51PNx+zLJO56cHw+xfWaB4snz8QEU\n5Puh3o+z+zuTjKEPCjWaSNIxwGe2T8seF3I0kcSCwCWkZLqrzau1v6ZSkpGmIe1zsD/27bW+bhF9\nVxYhyqJMlEVJww0tlTQF0M/2p0pr6twLHGf73uz5QiUDiYmBg4DDgOOBc2zqu3OYdDXwGfbP6nrd\nEELD6Mt3Z97NRDMDt0pqi+WatkRQNBJLApeSmqqWs3krhyC2AZYDlq77tUMITS3X0US237I9JLst\nbvs3ecbTEYlJJY4H/kwaxrlWHolgTWkr4CxgW+zPu3t9M4vx5CVRFiVRFpXJu2ZQaBIrkpaZfgNY\n0uadnALpt18aPXQqdr1GK4UQWkihOpDby6vPQGIAcAKwNbA/cGNVl5nufUCHA+sCa2LXt48ihNBw\nGrHPoHAkfgxcDDwKLGHzYc4BDQEOBpaNRBBCqJWizEDOncRAiYuBy0nLTO9QgEQwGWmW8UGCeXON\npUCibbgkyqIkyqIykQwAiY2Al4BvSQvL3ZVzSG1+BbwCXJN3ICGE5tbSfQYSMwJnk4Zr7manmbyF\nIK0BXAUsif2fvMMJITSOhlzCOg/ZwnLbkpaZ/hdpC8oH8o2qTJplfDmwWySCEEI9tFwykJgDuB0Y\nCWxkc4hN0cbtnwncjX1P24FoDy2JsiiJsiiJsqhMyyQDiYkk9gCeA54GlrEp3mY60kbAqsAheYcS\nQmgdLdFnIDEfaWG5AcBwm5cqDq4WpBmAF0jbVz6UdzghhMYUfQbtSPSTOAh4ArgTWKnAiUDAecB1\nkQhCCPXWtMlAYnHgMWAjYEWb0+q+wmjvbEnaLvPnHT0Z7aElURYlURYlURaVabpkIDGJxNHAKNK6\nQmvYvJlzWF2TZiYtQrcT9hd5hxNCaD1N1WcgsRxpmel/Anva/LNmwVVLah66Cfgb9hF5hxNCaHwt\nuzaRxOTAccBOpM1nrs11Ybne2QpYGNgu70BCCK2r4ZuJJFYDngfmIi0sd03DJIJS89Au2F92/dJo\nD20TZVESZVESZVGZhq0ZSEwNnAT8BNjb5racQ+qd1Dx0PnAZdvHmO4QQWkpD9hlIrA9cQNoz+RCb\ncXUPrlLSlsCxwNLd1QpCCKE3mr7PQGJ60lINq5Amj/0555D6RmpbIG+TSAQhhCJoiD6DbGG5LUjL\nTH9I6htozESQnAVcg/3Xnr4h2kNLoixKoixKoiwqU/iagcSspJm5CwGb2Tyec0iVSWsPLQ8MzjuU\nEEJoU/g+A/AHwIXACTZf5R1TRaSBpNrNdtgP5h1OCKE59aXPoBGSwVI2o/OOpSqki4FvsH+Wdygh\nhObVlAvVNVEiWB1YFzi8b2+P9tA2URYlURYlURaVKXwyaArS5MDFwF7Yn+QdTgghtFf4ZqJa7oFc\nN9JJwNzYW+cdSgih+TX9PIOGJC0N7ExanjqEEAopmolqSZqYtMPaodjvV3aqaA9tE2VREmVREmVR\nmUgGtbU/8B/g6rwDCSGErkSfQa1Ig4CngRWw/55vMCGEVtKUQ0sbUmlF0lMjEYQQGkEkg9rYCpgd\nOK1aJ4z20JIoi5Ioi5Ioi8rEaKJqk6YDzgA2xf4673BCCKEnos+g2qSLgPHY++QdSgihNcU8g7xJ\nqwAbAIvmHUoIIfRGrn0GktaV9Jqkv0nq05o9hSH1J+2+diD2f6t/+mgPbRNlURJlURJlUZnckoGk\nfsA5pMXbFgW2kbRIXvFUwUHAWODGGp1/SI3O24iiLEqiLEqiLCqQZzPR8sCbtscASLoe2Bh4NceY\n+kaaBzgUWJ7adcIMrNF5G1GURUmURUmURQXybCaaHfhn2eOx2bHGkuYU/BY4Hfv/8g4nhBD6Is+a\nQXGHMfXOxsC8wGY1vs6gGp+/kQzKO4ACGZR3AAUyKO8AGlmeyeBfwJxlj+ck1Q6+J+121hC+QrUd\nBStpp5peoIFEWZREWZREWfRdbvMMlFb0fB34MfAO8CSwje3G6zMIIYQGl1vNwPY3kvYB/gT0Ay6N\nRBBCCPko9AzkEEII9VHYheqaakJahSSNkfSCpOckPZl3PPUk6TJJ70l6sezYdJLuk/SGpHsltcSQ\nwk7K4lhJY7PPxnOS1s0zxnqRNKekUZJelvSSpP2y4y332eiiLHr12ShkzSCbkPY6sCapo/kpWrg/\nQdJbwDK2P8o7lnqT9CPgM+Aq20tkx04GPrR9cvZDYVrbR+QZZz10UhbHAJ/aPj3X4OpM0izALLZH\nS5oSeAbYBNiFFvtsdFEWW9KLz0ZRawbfTUhzWvmzbUJaK2usBfuqxPbDwMftDm8EXJndv5L0wW96\nnZQFtOBnw/a/bY/O7n9Gmqw6Oy342eiiLKAXn42iJoPmmJBWPQb+LOlpSbvnHUwBzGz7vez+e8DM\neQZTAPtKel7Spa3QLNKe0q6CSwFP0OKfjbKy+Gt2qMefjaImg+K1XeVrFdtLAesBe2fNBQFwauds\n5c/L+cA8pHV53qWKGyo1gqxZ5GZgf9uflj/Xap+NrCxuIpXFZ/Tys1HUZNCjCWmtwva72b8fALeS\nmtFa2XtZOymSZgXezzme3Nh+3xngElros6G0UvDNwNW2/5AdbsnPRllZ/K6tLHr72ShqMngaWEDS\nIEmTkLaRvD3nmHIhaQpJU2X3BwBrAy92/a6mdzvQNtN0J+APXby2qWVfeG02pUU+G0prgl0KvGL7\nzLKnWu6z0VlZ9PazUcjRRACS1gPOpDQh7Tc5h5QLpRVRb80eTgxc00plIek6YDVgBlIb8NHAbcDv\ngbmAMcCWtsflFWO9dFAWxwDDSM0ABt4CRpS1mTctSUOBh4AXKDUFjSStZNBSn41OyuJIYBt68dko\nbDIIIYRQP0VtJgohhFBHkQxCCCFEMgghhBDJIIQQApEMQgghEMkghBACkQxCi5J0nKTJ844jhKLI\ncw/kEKoqm6A3W9mhzYFPgXvLjv0HWAaYCVg8Ww67zfTArLZn7OT8swADbP+9g+fmB35ue+fs8anA\njbaf6Cbmn5KWJx/ZwXPzAv9rhUlkIX+RDEIzGQjMkd0XMA3Qv+wYwOrAusBPbH+cPUbS9MAdwBYd\nnTjbs/sKYHi74ycCqwCTAXNKejh7ahCwpqRPgddIM4bX6eDU0wJTSFoTeNr2z8qe+wK4UtIGtr/t\n7o8PoRIxAzk0FUm7AruRNoEZQJqK/zkwBWlfjDmBk4DzgKNtvy5pZeAa4FrbR3Vy3u2BuW3/qpPn\nZwI2I60lPx0wOfBE+1qEpElsjy97vDmpZnBkJ+c9Ehhr+6oeFkEIfRI1g9BspgRGkdZzL9/YYzAw\nte3DJK0DfAIMkHQN6Yt7Q2A/SXeSlgBu3xS0NbBvRxeUdAKwATCO1Pw0MfAOcKCkv9reN3vd2qQl\nyDe1PaH8FF38PdcB5wKRDEJNRQdyaEbfZrdvslvbYyTNSWruuZ7U5HOO7c1sv2x7BGnN9107OOe8\ntt/q4poH2F4d+A1wRnb/exsR2b6XtGnThT39Q7JrztPT14fQV1EzCM1GwE+AFdsdnw64BbgaeJyU\nJEYBv04rADMraQMQgEM7Oe8PD0q/JW21uJGk/5LtrCVpO1Iz1cySBtreIXvL/sCdkha1/Uqf/sIQ\naiCSQWg2Jm3y8Sjf/wJfCpgU+CmwN4Dtm7PXIulu2+t1cd4JHR3MmoDamoHmIjXnfAzsa/sHGzJl\nHcFdXed7JPUjq9WEUEuRDEKzEekX+bTZfWf/TgWMt/1hVhPorWclLWv76R9cMG06NJy0fvxOwFfA\n1ZJuAK7vZj392Umd3Z1ZFniqLwGH0BuRDELTkHQesAYwnjTcs9yUpA7jJYGXstePKnt+QNnjt2wP\nb/f+C4E9SSOVyq95OLAlcCWwmu2vs+PrALsAt2adyCPL3nMwcBhplNNXpGatzoygF30MIfRVDC0N\noYckXUDaY/aRsmOT2v6qRtdbDdjK9l61OH8I5SIZhBBCiKGlIYQQIhmEEEIgkkEIIQQiGYQQQiCS\nQQghBCIZhBBCIJJBCCEE4P8BC7nOdvSKp04AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xdc7e48>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[6], [8], [10], [14], [18]]\n",
      "[[  1   6  36]\n",
      " [  1   8  64]\n",
      " [  1  10 100]\n",
      " [  1  14 196]\n",
      " [  1  18 324]]\n",
      "[[6], [8], [11], [16]]\n",
      "[[  1   6  36]\n",
      " [  1   8  64]\n",
      " [  1  11 121]\n",
      " [  1  16 256]]\n",
      "一元线性回归 r-squared 0.809726832467\n",
      "二次回归 r-squared 0.867544458591\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "X_train = [[6], [8], [10], [14], [18]]\n",
    "y_train = [[7], [9], [13], [17.5], [18]]\n",
    "X_test = [[6], [8], [11], [16]]\n",
    "y_test = [[8], [12], [15], [18]]\n",
    "regressor = LinearRegression()\n",
    "regressor.fit(X_train, y_train)\n",
    "xx = np.linspace(0, 26, 100)\n",
    "yy = regressor.predict(xx.reshape(xx.shape[0], 1))\n",
    "plt = runplt()\n",
    "plt.plot(X_train, y_train, 'k.')\n",
    "plt.plot(xx, yy)\n",
    "quadratic_featurizer = PolynomialFeatures(degree=2)\n",
    "X_train_quadratic = quadratic_featurizer.fit_transform(X_train)\n",
    "X_test_quadratic = quadratic_featurizer.transform(X_test)\n",
    "regressor_quadratic = LinearRegression()\n",
    "regressor_quadratic.fit(X_train_quadratic, y_train)\n",
    "xx_quadratic = quadratic_featurizer.transform(xx.reshape(xx.shape[0], 1))\n",
    "plt.plot(xx, regressor_quadratic.predict(xx_quadratic), 'r-')\n",
    "plt.show()\n",
    "print(X_train)\n",
    "print(X_train_quadratic)\n",
    "print(X_test)\n",
    "print(X_test_quadratic)\n",
    "print('一元线性回归 r-squared', regressor.score(X_test, y_test))\n",
    "print('二次回归 r-squared', regressor_quadratic.score(X_test_quadratic, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "效果如上图所示，直线为一元线性回归（R方0.81），曲线为二次回归（R方0.87），其拟合效果更佳。还有三次回归，就是再增加一个立方项（$\\beta_3 x^3$）。同样方法拟合，效果如下图所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEaCAYAAADzDTuZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XWYXOX5xvHvTbAQHIJLcIpbcUqgSFIguFOc4k6BwK80\nVChQ3J0QIMGDJ1iTEijuDi0EGqC4BQuQ+/fHe5YZlt3NysycM7PP57rmytie8+zLMM+e5zXZJoQQ\nQvc2Wd4BhBBCyF8kgxBCCJEMQgghRDIIIYRAJIMQQghEMgghhEAkgxB+QlLN/p+QdLyknp34ucmr\nEU/o3uJDFQpB0nTAGKCtiS9LAg9m95cHnmrh/iu295E0NXCt7U2z4+8H7AWsa/uTsvPuBPSwfUX2\nxfyIpL8AP2RvecD2e9l7n7C9Yhu/wwLAXGVPbQl8Adxd9txHtl+WtCMwG7CUpJPLXp8FmNN2b0kL\nAisDvYBPgXmBm4DjJJ1p+7myc/8FuA3oD2ycnbfJnMBRtm9pLfYQIhmEougBfGJ7HUnrABvYHijp\nMdu/bP5mSUfaPjm7f47tA5q9ZXPgH9nr/YDFge2BIZJ2tP25pPOAddNbNBfpS/d54AFgBmAYsGF2\njB7AIpIeaiX+Z4ALgXmaQsyOMUXZc02x9wP2AzbOEtM62fOzkL7Qt87e+gYpoXwCTAdMbfstSasB\noyQtQ0oSMwFrkBLJ1MDetp8oO98ercQcwo8iGYQimgr4rvmTkhYBDiZdPTxT9tLMkuaxPa7suf2A\nP0g6FpgbOMD2REl/BG6TdInt/ST9CXgnO94K2c9uTfoL+7Cmq4LsGPfb3ritwCWtAOwJjCclF5MS\n0TTAMNvXSvo7MAC4QNJxtl+RtDpwNTDU9ujscAOB9UlXPuOADyR9CrySHf8A4Ergpex3eAlYGDhX\nUvmVwSzAsW3FHUIkg1BE0/PTMkeTuUl/IY8DFpV0DrAq8DEwVNL0wCnA98CCwLSkL8h3gaVJX/i/\nBZquFAA2JX1h/x04D9gR+BvwNjBBkpzWbOkNrCrpsVZiXsP2hOyco4CHSVcHTZYhXSlg+/eSNgQ+\nB3pJuhroSSrvHCTpdlLSO5H0/+gU2b8nAkcD79m+RdJBwM3ZVc7rwCOkq4T9bT8h6ShgpO3yxBlC\niyIZhCJaHHi2/ImstLI08CLwa9KX9qfApcDewCBglO2Rkk4DLieVnR7M6umvZodaDvjW9lOS1gU+\nAy4BFgHuA0aQ/rqeHzgCOBTYMiu7zNpSsFlfwoSyp36g1OcAKSn8kP2LpHmBwcBOwO7AObabyk97\nS+oL7EFKao8AHwL/AdYGDgPOzDqRp8kSwXzAEsBmpOS3kaQvgT7A1tlVwiu292kp/hAgkkEoGEmz\nker9p2RPfZfV0pcj1c5nJCWA3UklFAO3Zu9dQdJawOHAcWWHnYn0pd/c7sC1pC/p80kdugOyW5Mt\nO/orAJuQrljKzQzcmN2/CniI9GU/CjhBEqSO3nez9xwJ7JD9HgsALwBf2D5G0ovAqdkxAI4hJaqD\nSSWuw2yPy0pit5f3H4TQmkgGoShMGup8JXCw7a+y588Bbif9dTyYVCp6j/SFPdH2Bi0dLPtybRqZ\ntCylK4Nyvyd1IE9u+z1Jv7C9VtkxLgd6SrqJNJKnpZiP56flIJO+9B9s9vzypL4QgK1IfRrYvjF7\nP5JG2O5fdv6XSSOsFsjOf1T20nnA08Bi2TH2kfRn4MvsfXtLWp9Uhlozu3L4ne1/tvA7hABEMgjF\nIWAi0N/2xKYnbQ8FhgJk5ZMpSV/gN1EaddPqMbOfecf2t81ftP1uljTaZHuLNk6wGPC/Zr9HL9LV\niEjJQaS+jgnZ8T5o53m/kDSQVL7qDWyQzYM4itQncaik/bM+jclIfRLfkK449m+6IpB0+CRPFrq9\nqieDrD46hHQJbuAi22dJGkQadfFB9taBtkdWO55QWN+TSkBPtPFFOTWpL+B00hDMPdtx3KOA/cse\nT+Dncxl+fCxpTNnzCwN/bH5ASccD2wLfkkYJHZA93zRUdQJpqGe5aUmdxcvY3rbsWKPK3tOr7PEb\npP6LHYB/ZbeDgUWBf9neUdL5wA2S3iSVzAScRRqqWj6iaF6gtY7vEABQtTe3kTQHMIftpyVNCzxB\n6ujahlQDPa2qAYRQpyT9EnjZ9hdlz61s+9Gyx/PZfiu73zTyKYQOq/qVge3/kV1G2x4v6SVS3Rd+\nWlMNIZSx/bO/5ssTQfb4rbL7kQhCp9V0bSJJfUgdaQ9nTx0o6RlJl0qasZaxhBBCKKnlolzTAjeQ\nRoqMJw3lW4A0ZPBd0lC5EEIIOah6nwGApClIwwNH2D6jhdf7ALfZXrrZ83HZG0IInWC7Q2X4Wowm\nEmmS0IvliUDSnLabJthsDjzX0s939BdqVJIG2R6UdxxFEG1REm1REm1R0pk/pGsxz2AN0rT7ZyU1\nLTN8DLC9pOVIw/reIC0pEFrXJ+8ACqRP3gEUSJ+8AyiQPnkHUM9qMZroAVrumxhR7XOHEEJon9jp\nrH4MzjuAAhmcdwAFMjjvAApkcN4B1LOadCB3VjaHJvoMQgihAzrz3RlXBnUiW2MnEG1RLtqiJNqi\nayIZhBBCiDJRCCE0migThRBC6JRIBnUi6qEl0RYl0RYl0RZdE8kghBBC9BmEEEKjiT6DEEIInRLJ\noE5EPbQk2qIk2qIk2qJrIhmEEEKIPoMQQmg00WcQQgihUyIZ1Imoh5ZEW5REW5REW3RNJIMQQgjR\nZxBCCI0m+gxCCCF0SiSDOhH10JJoi5Joi5Joi66JZBBCCCH6DEIIodFEn0EIIYROiWRQJ6IeWhJt\nURJtURJt0TWRDEIIIUSfQQghNJroMwghhNApkQzqRNRDS6ItSqItSqItuiaSQQghhOgzCCGERhN9\nBiGEEDolkkGdiHpoSbRFSbRFSbRF10QyCCGEEH0GIYTQaKLPIIQQQqdEMqgTUQ8tibYoibYoibbo\nmkgGIYQQqt9nIGleYAgwG2DgIttnSZoZuBaYHxgLbGP702Y/G30GIYTQQZ357qxFMpgDmMP205Km\nBZ4ANgN2Az60fbKko4CZbB/d7GcjGYQQQgcVsgPZ9v9sP53dHw+8BMwNDACuyN52BSlBhFZEPbQk\n2qIk2qIk2qJratpnIKkPsDzwCDC77feyl94DZq9lLCGEEEomr9WJshLRjcDBtr+QSlcwti2pxXqV\npMGkPgWAT4GnbY/OXuub/XzDP7Y9ukjxxOPiPG5SlHjyetz0XFHiqeXj7P6uWTOMpRNqMulM0hTA\n7cAI22dkz70M9LX9P0lzAqNsL97s56LPIIQQOqiQfQZKlwCXAi82JYLMrcAu2f1dgJurHUs9i3po\nSbRFSbRFSbRF19SiTLQGsBPwrKSnsucGAicC10nag2xoaQ1iCSGE0IJYmyiEEBpMIctEIYQQii+S\nQZ2IemhJtEVJtEVJtEXXRDIIIYQQfQYhhNBoos8ghBBCp0QyqBNRDy2JtiiJtiiJtuiami1HEULI\nj8SspOXi581uMwPTAD2BqYBvgK+AL4EPgTdJ83/etPkqh5BDjUWfQQgNRmIWoC+wMrAssBzpC38s\n8N/s9hHpy/8rYEL2eq/s1hvoQ0oe82U/92h2GwM8Z1PcL45QzP0MuiKSQQiTJtEDWJO0LPy6wILA\ng8BDwFPAM8C4znyBS0wOLElKLCtnx29aa+x24B6b7yrwa4QKimTQwMpXY+zuoi1AYjKgL1x1COy0\nCvAOaX2vu4HHq/UFLSFgMWBj0h4kC5N2MrzU5pVqnLP9scXnoklnvjujzyCEOiIxD2mp4t2B8fDp\ng8DqNv+pxfltjPQacCVw19kcsMwQdt78BZZ8ZAG997+DOOu+gzjr7R5MnAroATR9IU0klaO+zf79\nAvg8u30KfJDdxlPkv1AbWFwZhFAHJFYFDgd+DVxHWgn48arU7qUpSaWmRYFFKPUdzEfapXBm0hf4\n+8DHwGffMNVnJ3L0nBewz3LT8NXX+3He6EM5/eUeTDRp7/MewJTZbWpgWmA6YHpgJlI/xWykEY7v\nkPo1xpE6sv8N/Cf7991IFpMWZaIQGkhWkhkAHAXMAZwBXG7zRYVOINIX/YrA0mW3+Uhfxq+SvoDH\nkr6U3wLeBj7A/r6VmHsAWwLHAR/A+h/BvbOSOqp3sP3pJGLqRUo485BGPc0PLEQqRy1CSiYvAs8D\nzwFPAk9jf9mZJmhUkQwaWNRDSxq9LbIksAkwiFRm+Ssw3OaHn7+3A22RdhtcldTZvAqwEvA98Dip\nk/m57PYadpf6HLKO573g4zPh2ingD8BH19netivHRZqV1KG9JLAMsEJ2/82hMHYHuAn4F/Ay9sQu\nnauORZ9BCHVOYl3gJNJfwIOAmztdCkp/Za9FKi2tAyxOGl30AHAB8Bj2O12P+udsvgfOl5bcAo5Z\nD176Dr66tQIH/hD4Z3ZL0k6KS74LOwNrk/ZLmRHpn8Co7PZClJfaFlcGIRSAxOLAycBSpC+z6206\n9pdtKvssAfwmu/0SeAK4D/gH6cv/2wqG3Z6QZgQuhGcvhaXPBR4GDrL5pMonnps012Id0nDYqYCR\nwAjgXiZVrqpzUSYKoc5IzAD8CdiRtPvf2Tbt/8KWepB2E9wC2JTUAXsHcCcwqki1dIlepN9xM2AH\nmzE1PPkiQD+gP6lM9ghpKO4t2ONqFkeNxEJ1DSzWXSlphLaQkMT2pM7QaWCBO0Ebg4Znf0239cOT\nIfVFuvCetHTEmaQZxQOAPtj7Yd9epEQAYPOlzYHAXsANEgdn/SMV0ebnwn4N+2zs3wBzAueT+k2e\nQXoI6ZDsaqLbiiuDOtHonaYdUe9tIbEQcCEwK7CvzUOSRpPq3QAtd7RKy5Lq4tuRxuQP6w9vjbCH\n1STwCpJYALgReBnYy6bLiatTn4vU3/BrUpsOAJ4lTaK7AfvzrsaUlygThVBg2bDLA4H/A/4GnJl1\ntCLpTlIJ4zFggx+HYKbRMzsAu5HG9w8BhmK/VPNfoMIkegLnAcsDv7GpSmd2BwKaitTXsjOpr+EO\n4DJSua2uRiZFMgihoCQWBS4nzcTd3ea1n77e1NHK3obPSHXtfUhfTndkP1t3X0qTkpWJBgK/A/rb\nFCPJpSS8Pamk1RO4GBiM/X6ucbVT9Bk0sEaok1dKPbVF1jewD2ns+7XA2s0TAYDtTw27O30BPQ9c\nRLpKWAh7J+z7WkoE9dQWLbGxzQmkYbSjJdbo7LEq2hb2h9hnk1Z93Yk0LPcVpKuQVq7YeQok5hmE\nUCUSs5GWjZgLWNPm5VbeOB9wAGm9oTHZ/dHdaVy8zWCJd4HhEjvbjMw7JoDsv8EjwCNIR5D+G12L\n9D5wOqlvocXZ2PUmykQhVIHEBsDg7DbIZkILb1oG+D2pFHQFcA7267WLsngkVgNugQIlhObScN6N\ngcNIy2WcDlyKPT7XuMpEn0EIOcuWYRhEWln0tzajWnjTGsCxpE1nzgQubPRJUB1RFwmhibQKcARp\ngts5wFnY1Z1Q1w7RZ9DA6r02XElFbQuJuUmzfVcGVvxZIpDWRrqPtPzzzcCC2Cd1JREUtS26wuYh\n0gS6IRL92vtzubSF/Qj21qSJf/MD/0Y6Aal3zWPpokgGIVSARF9Sh+89QD+b98peXJM0j+ASUiJY\nDPsi7G9yCLUulCWEK7MrhWKzX8XenbQC7EzAy0h/QZop58jaLcpEIXRBNjTyENIy07+1uafsxRWA\nv5DWCzoeuLJROhtrRaI/aVht31Y74ItI6kOaT7IZqRR4ei37FKJMFEINZWvtDCUNPVz1x0QgLYh0\nDWmP4DtIVwKXRyLoOJsRpEQ7UmKuvONpN3ss9p6kJcN/AbyKtG8247mQIhnUiUasDXdWEdpCYj7S\nUtATSMNGxyLNgnQ6qVz0ArAI9rnVXCm0CG1RbTZXkNYSGpkt7NeiQraF/W/sHUijj7YAnkfaLFth\ntlAiGYTQQdnEqIeBq4Bdjb5HOgh4ibRU8hLYfy7aQnF17mRS8h2aLetRX+wnsdcnLUfyF+C+bK2p\nwog+gxA6QGJX0hfTLjYjkDYkjTMfBxyK/UKe8TUyiSmAu4GHbQbmHU+nSdkucAwiDaE9FvuDyp4i\n+gxCqAqJySROIO3fuLbRi0jDgXOBo4ENIxFUl813wNbAdtny3/XJ/h77fGAx0t7QLyDtk01my00k\ngzpRyHpoTmrdFtnqmsOAtVfisbWMNiXtIPYksBT2rXktHdHdPhc2H5JG6JwlsWL5a3XXFvan2IcA\n65E2N3o0z3WPIhmE0AaJ3qSJZB7L/H94jJXvJa0ounLWLxBzBWrM5hlgX+BGiboZx98q+1ngV8AZ\nwC1IZyFNX+swatJnIOkyYCPgfdtLZ88NAvYkbdIBMND2yGY/F30GITfZJjQjp+PzWz9m5l6T88PG\nwEHA8O60iFxRSZwJzAdsYdMY/z2kmYG/AxsAB2EP79xhittncDn8bFq5gdNsL5/dir0GSehWJH4J\nPLApN9/zOTNsOzk/QCoJ3RSJoDCOBOYlrfLaGOyPsfcgzV05Eel6pNlrcepJJgNJPSTtLOlOSS9l\nt+clXS5p1facxPYYoKXFm+Kv/naqu3poFVW7LSR+IyaOOI99X72ZzdcHdsDep4iLyXXnz4XNt8C2\nwB8kVmyotrD/SVrI8HXgWaQdqj03oc1koJSR7gDmAfa3/QvbvyBt+HAB8DtJ53Xh/AdKekbSpZPc\nBDyEGpDYeWq+vno0fb0vFzwBLIt9f95xhZbZ/Id0ZXAtLDRN3vFUlP019lGkCWvHAMORZqvW6drs\nM5C0IPC17XfbeM/aTlms7ROltTpuK+szmI1Sf8GfgTmdLo/Kfyb6DELNzKyPj5mMicfcx68/WpZn\nd8R+IO+YQvtIXAxMZrPHJN9cj9L+zINIe2Hvh31T22/vxHens33nqn0D+gDPdeQ1Ur/C4KwRBpEW\nBOtb9nrfeByPu/oYrGV4euhcDPnuROa7xzBtkeKLx5N+DJ4ORr4LRw8sQjzVerw37G94zTBk0TQo\np2/ZewdT+r5008+199ahN3flRrMvfNKVQNP9Q4GhLfxMh3+hRr2Vfyi6+62SbXEtW0+9Lvc+vTxP\nTHiQ1bbP+3eLz0VX2uKAg8Fvg2fJO5aq3qCX4QLDG4bVW/lcuKPHrcloIknDSBuCLybpv5J2B06S\n9KykZ4C1s4QQQs28qfl/MYSd3/mEmebZl/MXXd3/GpZ3TKErznkGuJY0K7xx2V9i70P6zhyONChb\n4qJL2j3PQNK5tvfP7p8M/BK4xPbVXQ2ijXPa0WcQquA9zb7nLlxx7lj6vDKeaVce53li8lgDyGaL\nP0nad/ravOOpOmku0v7ZU5NGvf03PV3hPZAlXWV7p+z+Q7ZXK3ttAWCk7cU68Su0L7hIBqHSpF4f\nMOuFA7h189dZcPT7zL6Z05o3oUFIrAzcCizltHxFY5MmI825OATYA/uOakw6W6i1F2y/Ad2goQui\nocZQd1Gn20Ja8iNmfmItxmz4FMtf+z6zD6j3RBCfi5KmtrB5FLiGNJO38dkTsU8EtgLORzqlM4fp\nSJ1paknXN/vZXp05aQg1J/32fXqfvhTPj/+A3kNBh6S+uNCg/gC8ILGOzai8g6kJ+wGk5Ullow6b\nVJnox9KQpIeBXSjNGjbwlu2vO3PidgUXZaLQVWl89unjmHvDRXht4jf0vB44NhJB45MYQLo6WNam\n+/QJSZMJfqh0maicbb9i++Xs9ko1E0EIXSbNC9z/Ggv3mZ83f/iGnkNsjolE0D3Y3ErafrR+N8Lp\nDHtiZ36sI8lgmmxNon9JOiObURxqJGrDJe1qC+lXwKMPs8q9i/LqIhPpcZnNn6seXI3F56KklbY4\nENhPYvEah1N3OpIMvrK9FGljiWeBuyWtUJ2wQugkSUj7A9cNZfsjVuPh7UHn25yYd2ih9mzeBk4A\nzpBiYcy2TKrPoHxuQfOhpcsAQ4EVbE+oSnDRZxA6IvUPnAesPJAT9j+RgUOA02zOyjmykKNs7+Rn\ngSNtbss7nlrozHdnm6OJmhJB5p/NXns2W7F0VuCdjpw0hIpLCx/eBLy3JmO2f5A17wBOsunKqrqh\nAdh8J3EIcK7E3U5LX4dm2rOfwRFKU52Pk3RO9lxT7XUcMGcV4wuZqA2X/KwtpKWBR4BR0/LFUVki\nOLE7JIL4XJS01RY2dwEvkiZmhRa0Z57BRqQNFn4AlsyeWzP7dx3STL8Q8iFtRNpJ7xDhR4BRwAk2\nF+QbWCigw4GHJIbYtLosf3fV6pWBpMUlbdz0kGa7kinturMyEGu+14Dt0XnHUBQ/toV0IHAxsGlZ\nIvhrd0oE8bkomVRb2LwGXAr8rSYB1Zm2rgz+C5wPTEGaYNa8p/kQ4ArbdT2dP9QhqQdwCrAhsIaw\nKF0RXJhrbKHo/gq8KrGszTN5B1MkrV4Z2P4S6A/MkP37G6C3pO2A5YFZbF9UkyhD1IYzM0mXngIf\nPwG/PRH6K/2N8g9SH0G3uSJoEp+Lkva0hc3npIQQQ42babMD2fY3wETgG+Db7P5XpCuKeZRWywuh\nNqTe98PWH8L0q8EsA1n6HFIi+LvN+XmHF+rGhcCiEuvmHUiRtDrPQFJP0sp/MwBnkzqQD7a9jqRR\nwDBgftvHVi24mGcQmqQl0++6CCbbGxaChZ+Bl2eEHjGPIHSYxHakDuWVG3F5kkovYb0UcDql/oKf\nNFhWIvqVpBk6GmgIHZJmuj8AnLE3rASL3QYvTw89zoxEEDrpOtKgmK3zDqQo2uozeKysd35idmvu\ndmD9KsQVmum2tWFpHWAkcCD2eeBp4Izlocf5NqfnHV7euu3nogUdaQubicBRwAnZDOVurz01/2tt\n32z7NuC57LmR2b+jgc+rEVgISFuQ9rTdGvsmidmB++D1EXY32bgkVI3NfaQ5VHvkHUsRtHsP5DxE\nn0E3Ju0B/BnYGPtJid6k4aPX2fwp3+BCo5BYBbgeWKSRlqmoxraXIdSedDhpp6q+WSKYBbgXGB6J\nIFSSzSPA88TVQSSDetEtasNp+enjgb2ANbFflZgJuJtUmjwue1vf/IIslmiLki60xSBgoMTUlYum\n/rSZDCQtJGmuSbxnzbZeD6Fd0vImp5L2y/gV9jiJGYC7SCvmHt2IQwBD/mweBZ4B9sw7ljxNaj+D\nOYDBwBhgqO03yl5bCdgH+N72PlUJLvoMuoc0efF8YDmgP/bHEtORrgaeAg6MRBCqSWIl4GZg4UbY\nL7kz352T7EDOlq/eLrstSBqbOxF4ErjQdtUWqotk0A2kdYYuBRYgdRZ/IdELGAG8BOybDQMMoaok\nbgXubYS5K1VJBnmKZFAiqW/DrVCZ/tAYTNoTYwD2lxI9SfNX3gL2aCkRNGRbdFK0RUlX20JiBeA2\nYMF6H1kUo4lC/ZCmAK4CZgM2yRLB1KRL9XeBPeOKINSSzZOk7TF3zjuWPMSVQai9lAiuBqYFtsD+\nRmJK0raVXwI72nyfZ4ihe5L4FalsubjND3nH01lxZRCKL5WGruKniWAK0kzjCcBOkQhCjsYAHwBb\n5h1IrU1qaOnskmbP7kvSjZJmkzR/dpuvNmGGhhhPnhLBlaSVcJsSweSkq4TJge1sJrlZUkO0RYVE\nW5RUoi2yUWt/A46W6FZViUldGawJ3CVpXWAN4CPgRtIkjUGkTchDmLQ0augKYGZgsywR9ACGANMB\nW9lMyDPEEDJ3kHZ43CDvQGppUvMMtgReJC1lPSOpY+VS22tlr49pul+V4KLPoDGkeQSXAvOSOou/\nzhLBZcBcwACbr/MMMYRyEjuRRrOtk3csnVGNPoMewGekMd9zA292MrbQXaWZxecACwObZolgMuAi\nYD5g00gEoYCuAfpIrJp3ILXSajLItrT8P9LG472BgcBBNYorNFOXteHSEhMrAhtlw0cnAy4AFgE2\nsfmqE4ftW9E461i0RUkl2yIbxHAacFiljll0bW1uMxFYhbTe97ukTr7+wHOSRmVbX8aoj9CWPwHr\nAP2wP8865M4BlgQ2shmfa3QhtO1yYF2JPnkHUgvtWY5iIGkTmwVJf82db/u96ocWfQZ1TTqCtCzw\nr7A/yBLBmcDKwAZ2bIoUik/i78BkNofnHUtHVHw5Ckm7AMeQOpD3IV02lZ/Atoe0I7DLgI2A920v\nnT03M2ls+fzAWGAb25929RcKBSDtRfrcrJWtPirgDGA1YH2bz3KNL4R2kpiPtFhiH5sv8o6nvarR\ngfwDYNKID5GWeJ2WVB76Pnu9PS4H+jV77mjgHtuLAvdlj0Mr6qY2LG1LGna8QVkiOB1YnXRF0OVE\nUDdtUQPRFiXVaAubt0gbK+1e6WMXzeRtvWj7KkmrA68CuwIzkRrl97bfbe9JbI+R1KfZ0wOAtbP7\nV5BKUZEQ6pm0IXAWsD72a1kiOJU0X2V9m0/b/PkQiul0YKjEOfW8RMWktKfPYHrSSn794MdJQU0/\nZLdzcaMsGdxWVib6xPZM2X0BHzc9LvuZKBPVC2kV0mqjm2E/mCWCU0gJf32bT3KNL4QukHgI+LvN\nTXnH0h6d+e5s88pA0kykGaKHAieQNh8pNwHYsCMnbIltSyruinmhbdIvgFuAXcsSwWnAWqTSUCSC\nUO9OI30P1kUy6Iw2kwHwD2B72y+TNrOppPckzWH7f5LmBN5v6U2SBpM6mAE+BZ5uWrO8qUbYHR6X\n10OLEE/T477Qe1QqBR0p+BL16As/bAasAUsOgheXgcqev3mbFKk9cni8nO0zChRPno8PoXrfD8Ph\n7nOlK/awr760IL/vj4+z+7uSjKUTJjWaaD9gceAQYGnSGjKfAa/a7tDmDy2UiU4GPrJ9kqSjgRlt\nH93sZ6JMlFERNzFJV44PAJdjn1I2fHRV0hVBVfoICtkWOYm2KKl2W0j8AZjHZu9qnaNSKj60NDto\nP1Jn7+OkuQYzAcsC7wBH2369HYENI9WOZwXeA44jlRWuIy1JMJYYWlpfpKmBu4HHsQ/LZhafQ5pt\nvGF0FodGIzEHaSvWPkUfHl2VZJAd+FjgJds3lT3XFzjKdv+OBtru4CIZFFNagfQa0l7Y2yuNJ7gQ\nWALoHxOhYyqiAAAUSUlEQVTKQqOSuAZ40ObsvGNpSzUmnd1C6jd4hjS/oKkj0KR5B9h+sFPRtie4\nSAY/Kkw5II38OgNYBugn/D2lDe1rssREYdqiAKItSmrRFhJrkRZZXCLb+6CQKj6aCNgW2Bz4M6lE\ndDn8bPORqiWDUEiHAOsCawlPJG1WMzvwG5svc40shOp7gDThdh3SH8oNo917IEvajrSuzOFu7w91\nUVwZFIy0BWlS2erC7wHDgKmBLWMZ6tBdSOwLrGcXd2vMqvUZ5CWSQYGkSWW3kUpDL5F2vPsG2N7m\nW0kXAYsCXwE7NB8MEEKjkJiOtLfLMjbj8o6nJZ357pzU2kShIHJdg0ZaEBgO7Cb8Kmmm8SfANjZN\nQ4wXJY0Y60/qTK5iOLEeT5Noi5JatUW2YN0w0lptDSOSQWibNCNpT9i/CD9EWrTrdWDnbAOQJk2b\n1DwGxR+HHUIXXQzsnm3f2hCiTBRaJ01B2vL0eeETSfMK7gKObD6SQilpXAjsHSWi0B1IPAb8wWZk\n3rE0F2WiUDn6ce/ib2flg7OAMaT9J36WCABsf2p720gEoRu5hAYqFUUyqBM51IYPBVZbkceP/4hZ\n/wmcZfPXIoytjjp5SbRFSQ5tMQz4tcTsNT5vVUQyCD8nbQwc8RvuGPQkK94GHFX0GZch1Fo20344\nsHPesVRC9BmEn5KWBEbtwNUnDWOHo4HfFrEmGkIRSKxOmoy7eBGumptEn0HoGmkW4NbduOzmYezw\ne2CTSAQhtOkh0va/a+UdSFdFMqgTVa+HSlNMRDfsxJXvDGa39YG+Ng9X9ZydFHXykmiLkjzaIrsa\nuATYq9bnrrRIBgGAb5nyrK24YcGr2bEXsJrNy3nHFEKdGAIMkJgh70C6IvoMAm9q/gN24/K/PcCa\nj3zHlJtnMyxDCO0kMRy4w+aSvGOB6DMInTBM22+2Mbef/gJL3vkdU/aPRBBCp1wB7JJ3EF0RyaBO\nVKMeOkC3bng4p97Ymw+GvM/s29k/W568kKJOXhJtUZJzW9wJLC6xUI4xdEkkg25qRn267RjWun0v\nLh72D6+7R5GGxYVQb2wmkCah/TbvWDor+gy6GQmJiX+YmY+PvJodH96Qu9enyB+CEOqExEqkfd0X\ntpmYbyyV3+ksNBCJXsDlc/C/lR5m1bfn47+bRSIIoWKeAL4G1gTuzzmWDosyUZ3oaj1UYkHgwXn4\n77T/YaFp5+O/m2BXfb/iaog6eUm0RUnebZGVWq+gTpeniGTQwCRdJGm0tNNjMPHhuRl3/ZvMv3RP\nvtkT+9W84wuhAV0FbCExTd6BdFT0GTQwqcdoOGZt2IceHDDqe26eDLgf+7i8YwuhUUmMBIbYDM0v\nhphnEDISveGBJWBDYPWnP+bmZ4FvgeNzDi2ERncVsEPeQXRUJIM60ZF6qMSvgCdhyatgvevH8tYp\n08PmwI7YP1QtyBrJuzZcJNEWJQVqi1uAtSRmyTuQjojRRA1EYnJgILAfsJs9/Uj07cLAv4CNsT/M\nNcAQugGbLyRGAFuRtoKtC4XvMwAvYvPvvGMpumy00JWkoW272LyN1BN4GLgI+9xcAwyhG5EYABxu\ns3Y+52/MPoMxEr/MO4iikpDErsAjwA3ABjZvZy+fDbwAnJdTeCF0V3cBS0nMm3cg7VUPyWBv4E6J\nfnkHkqeW6qESc5Pqk4cBv7Y5/ceZj9LOpMkvezfaxLIC1YZzF21RUqS2sPkWuAnYLu9Y2qvwycDm\nVmBTYLBU/xtIVEJ2NbAb8FR2W8nm2bI3LAGcCmyNHauQhpCPocD2eQfRXoXvM2iqe0ksBtxMmuZ9\nUJZ5ux2JRYBzgVmB3WyeafaGXsCjwCnYl9c+whACgEQP4C1gPZuXanvuxuwzAMDmFWBl0pfg/RLz\n5BxSTUn0lDietOfqXcAqP0sEybnAY5EIQsiXzQ/ANdTJ1UHdJANIQ7ZIw7VuAh6T2DTnkKouKwlt\nBiP/DfwCWM7m1Bb3Hkj9BCsD+9c4zJoqUm04b9EWJQVti2HADhKFX0mhrpIBpMWgbE4CtgZOk7i8\n3vcebU02imo08Ge453SbbWzGtfLmxUj9BNtif1m7KEMIbXgCELB83oFMSt30GbT8OtMCpwD9gN/Z\n3F2z4Koo6x/5I7A2cBwwOLvkbO0HmuYTnIt9UU2CDCG0i8SJgG0G1u6cDdxn0BKb8Tb7APsA50sM\nzyZf1SWJxSSuAsYAzwOL2VzaZiJITgVeBi6udowhhA67Hti66KWi3JOBpLGSnpX0lKRHO3MMm5HA\nksBjpL6Ev0jMWNFAq0hiNYnrgQeAl0g7JZ1gM770nlbqodIWpNXoftdo8wlaU9DacC6iLUoK3BZP\nkr5rl8s7kLbkngwAA31tL2975U4fxHxjcwKwLDAv8G+JPxd1sSiJqSS2k3gIfrwaWNDmrzaft/Mg\n8wLnAztgf1a9aEMInZVtenMdqZ+zsHLvM5D0BrCS7Y9aeK3T+xlILERatG0LYAhwsc0LXQq2AiSW\nA3YnLXH7DHAOcGs7SkHND9QD+AcwAvvESscZQqgciRVJw0wXzZJDlc9Xn30GBu6V9Likis0wtvmP\nzZ6kS7MvgXskHpTYTWLmSp1nUrKhoStI/FXiJdLyEZ8Av7T5tc3wDieC5Bjge+DkSsYbQqiKJ4Ee\nFLhUVIQrgzltvyupN3APcKDtMdlrTXuKjs3e/inwtO3R2et9AdrzOC3vfOxRsHw/2GpZ4Ck4+0X4\n16MwbIjNDx05XluPwWOBvjBsO5hlBdjgM+Am2HMsDHnZnjCq4/GX6qGG74Ab14T9H4SPuhpvvT1u\n3iZ5x5Pz4+Vsn1GgePJ8fAid/H6ozePLhsLEifaeO1X6+Nn9XUnGAn/s6JVB7smgnKQ/AuNtn5o9\nrsq2lxI9gXWBAUBfYE7SeOBHgdeAN7Lbu8A3LV3WZcNa5wHmBuYjdWAvCyxDutoZDYzK/n21q5eG\nkvraHo00A/A0cDD2rV05Zr36sS1CtEWZoreFxEqkSWhVLxV15rsz12QgaRqgh+0vlNbUuRs43vbd\n2es12QM5Kxv9ElgJWBhYILvNQdoAaDzwVXZ/aqAn6a/zcWW3F0l9AM8A71TtP7Z0JTAee9+qHD+E\nUBXZ0NLXgc1tnq7uuTr+3Zn3TmezA8MlNcVydVMiqCWbj0nr/dzV/DWJKYBe2e074BvSBjLf16Ij\nqFkw25OS1go1PW8IoctsnA0h3wqqmww6o1BlouZqdWVQD9aTtr03bVbTD/vJvOPJU9HLAbUUbVFS\nD20hsRpwic2S1T1PfY4mCpMi9TgojR46pbsnghDq3CPATBKL5h1Ic3FlUA+ko0jrL62H3ZlhqCGE\ngpA4HxibLbhZpXPElUHjkZYDDgd2iUQQQkMYDmyedxDNRTIoMmlq0lIVh4n6XYCv0gq8Bk3NRVuU\n1FFbjAYWk5gr70DKRTIotr+ShqxenXcgIYTKsJkA3AnF2pwr+gyKSlqXtKbSsrSwblMIoX5JbEXa\ng2WD6hy/ziadTUq3TQZplvGzwN7YI/MOJ4RQWdkKBu8A89t8UvnjRwdyoziDtBrpj4mgjuqhVRdt\nURJtUVJPbZHtVTIa2CjnUH4UyaBopAHAr4Aj8g4lhFBVhRpVFGWiIpFmJZWHtsO+P+9wQgjVIzEr\naa2i2W2+ruyxo0xUv9ICTecBwyIRhND4bD4kLWy5Tt6xQCSDItkGWBr4v5ZerKd6aLVFW5REW5TU\naVvcBmySdxAQyaAYpNmBM0mzjCt6uRhCKLTbgI2z5a1zFX0GeUvloRuA17CPzjucEELtZEngNWBr\nm6cqd9zoM6hH2wKLA4NyjiOEUGPZniiFKBVFMshTqTy0G/Y3bb+1LuuhVRFtURJtUVLHbRHJoFtL\n5aHzgcuwH807nBBCbsYAC+e9cF30GeRF2oZUGlphUlcFIYTGJnENcJ/NxZU5XvQZ1AepN3AWsHsk\nghACBSgVRTLIx5nA1dgPt/cH6rgeWnHRFiXRFiV13hYjgL4SPfMKYPK8TtxtpbWHVgaWyTuUEEIx\n2Hws8RTwa+D2PGKIPoNakmYEngd2xP5n3uGEEIpD4ghgIZt9u36s2M+g2KSLge+xu/wfO4TQWCSW\nBO4AFsjmH3ThWNGBXFzSOkA/4KjO/Xhd10MrKtqiJNqipAHa4kXSd/Iv8jh5JINakHoCFwP7YX+e\ndzghhOLJrgbuBPrncf4oE9WCdBIwP/Z2eYcSQiguiU2BA23W69pxos+geKQVSMPGlsZ+P+9wQgjF\nJTEdaW/kuWy+6Pxxos+gWKTJgUuA33c1ETRAPbRioi1Koi1KGqEtsgTwCLBurc8dyaC6DgY+Aq7M\nO5AQQt0YQQ79BlEmqhapD/A4sAr2f/INJoRQLySWAEYC83d2iGmUiYqitCLpKZEIQggd9BIwEVii\nlieNZFAd2wJzA6dW6oCNUA+tlGiLkmiLkkZpi+xqoOalokgGlSbNDJwO/A77u7zDCSHUpRHAb2p5\nwugzqDTpImAC9gF5hxJCqE8S0wLv0skhpp357oxVSytJWgPYiBrX+kIIjcVmvMRjwNrUaBXTXMtE\nkvpJelnSa5I6tWZPYUhTABcAh2J/VvnDN0Y9tBKiLUqiLUoasC3uBjao1clySwaSegDnkBZvWwLY\nXlIuCzRVyGHAOOD6Kh1/uSodtx5FW5REW5Q0Wlt0j2RA2uDl37bHOnW0XgNsmmM8nSctAPwe2J/q\ndcLMWKXj1qNoi5Joi5JGa4ungVkk5q/FyfJMBnMD/y17PC57rr6kOQVnA6dhv553OCGExmAzEbgH\nWL8W58szGRR3GFPHbAosCJxS5fP0qfLx60mfvAMokD55B1AgffIOoApqVirKbWippFWBQbb7ZY8H\nAhNtn1T2nkZJGCGEUFN1s4S10oqer5A2gH4HeBTY3vZLuQQUQgjdWG7zDGx/L+kA4C6gB3BpJIIQ\nQshHoWcghxBCqI3Crk3UUBPSukjSWEnPSnpK0qN5x1NLki6T9J6k58qem1nSPZJelXS3pEYbUtii\nVtpikKRx2WfjKUn98oyxViTNK2mUpBckPS/poOz5bvfZaKMtOvTZKOSVQTYh7RVgPeBt4DG6cX+C\npDeAFW1/nHcstSZpLWA8MMT20tlzJwMf2j45+0NhJttH5xlnLbTSFn8EvrB9Wq7B1ZikOYA5bD8t\naVrgCWAzYDe62WejjbbYhg58Nop6ZdA4E9Iqp74W7KsQ22OAT5o9PQC4Irt/BemD3/BaaQvohp8N\n2/+z/XR2fzxpD4C56YafjTbaAjrw2ShqMmiMCWmVY+BeSY9L2ivvYApgdtvvZfffA2bPM5gCOFDS\nM5Iu7Q5lkeaUdhVcnrR3cLf+bJS1xcPZU+3+bBQ1GRSvdpWvNWwvT9rsYv+sXBAApzpnd/68nA8s\nQFqX510quKFSPcjKIjcCB9v+yVLP3e2zkbXFDaS2GE8HPxtFTQZvA/OWPZ6XdHXQLdl+N/v3A2A4\nqYzWnb2X1UmRNCfwfs7x5Mb2+84Al9CNPhtKKwXfCFxp++bs6W752Shri6ua2qKjn42iJoPHgUUk\n9ZE0JWkbyVtzjikXkqaRNF12vxdpavpzbf9Uw7sV2CW7vwtwcxvvbWjZF16Tzekmnw2lNcEuBV60\nfUbZS93us9FaW3T0s1HI0UQAkvoDZ1CakPa3nEPKhdKKqMOzh5MDV3entpA0jLTBx6ykGvBxwC3A\ndcB8wFhgG9uf5hVjrbTQFn8E+pLKAAbeAPYuq5k3LElrAvcDz1IqBQ0krWTQrT4brbTFMcD2dOCz\nUdhkEEIIoXaKWiYKIYRQQ5EMQgghRDIIIYQQySCEEAKRDEIIIRDJIIQQApEMQjcl6XhJPfOOI4Si\nyG2nsxAqLZugN1fZU1sCX5A2FW/yEbAiMBuwVLYcdpNZgDlt927l+HMAvWz/p4XXFgb+z/au2eNT\ngOttPzKJmLciLU8+sIXXFgS+7A6TyEL+IhmERjIjME92X8AMwBRlzwGsA/QDNrH9SfYYSbMAtwFb\nt3TgbM/uwcDuzZ4/EVgDmBqYV9KY7KU+wHqSvgBeJs0Y3rCFQ88ETCNpPeBx2/uWvfY1cIWkjWz/\nMKlfPoSuiBnIoaFI2gPYk7QJTC/SVPyvgGlI+2LMC5wEnAccZ/sVSasDVwNDbR/bynF3Aua3/ddW\nXp8N2IK0lvzMQE/gkeZXEZKmtD2h7PGWpCuDY1o57jHAONtD2tkEIXRKXBmERjMtMIq0nnv5xh7L\nANPbPlLShsDnQC9JV5O+uDcGDpJ0O2kJ4OaloO2AA1s6oaS/ABsBn5LKT5MD7wCHSnrY9oHZ+zYg\nLUG+ue2J5Ydo4/cZBpwLRDIIVRUdyKER/ZDdvs9uTY+RNC+p3HMNqeRzju0tbL9ge2/Smu97tHDM\nBW2/0cY5D7G9DvA34PTs/k82IrJ9N2nTpgvb+4tk51ygve8PobPiyiA0GgGbAKs2e35m4CbgSuAh\nUpIYBZyQVgBmTtIGIAC/b+W4P39SOpu01eIASZ+R7awlaUdSmWp2STPa/m32IwcDt0tawvaLnfoN\nQ6iCSAah0Zi0yceD/PQLfHlgKmArYH8A2zdm70XSCNv92zjuxJaezEpATWWg+UjlnE+AA23/bEOm\nrCO4rfP8hKQeZFc1IVRTJIPQaET6i3ym7L6zf6cDJtj+MLsS6KgnJa1k+/GfnTBtOrQ7af34XYBv\ngSslXQtcM4n19OcmdXa3ZiXgsc4EHEJHRDIIDUPSecC6wATScM9y05I6jJcFns/eP6rs9V5lj9+w\nvXuzn78Q2Ic0Uqn8nEcB2wBXAGvb/i57fkNgN2B41ok8sOxnDgeOJI1y+pZU1mrN3nSgjyGEzoqh\npSG0k6QLSHvMPlD23FS2v63S+dYGtrW9XzWOH0K5SAYhhBBiaGkIIYRIBiGEEIhkEEIIgUgGIYQQ\niGQQQgiBSAYhhBCIZBBCCAH4f1eFf25x2ufsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x9027c18>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[   1    6   36  216]\n",
      " [   1    8   64  512]\n",
      " [   1   10  100 1000]\n",
      " [   1   14  196 2744]\n",
      " [   1   18  324 5832]]\n",
      "[[   1    6   36  216]\n",
      " [   1    8   64  512]\n",
      " [   1   11  121 1331]\n",
      " [   1   16  256 4096]]\n",
      "二次回归 r-squared 0.867544458591\n",
      "三次回归 r-squared 0.835692454062\n"
     ]
    }
   ],
   "source": [
    "plt = runplt()\n",
    "plt.plot(X_train, y_train, 'k.')\n",
    "\n",
    "quadratic_featurizer = PolynomialFeatures(degree=2)\n",
    "X_train_quadratic = quadratic_featurizer.fit_transform(X_train)\n",
    "X_test_quadratic = quadratic_featurizer.transform(X_test)\n",
    "regressor_quadratic = LinearRegression()\n",
    "regressor_quadratic.fit(X_train_quadratic, y_train)\n",
    "xx_quadratic = quadratic_featurizer.transform(xx.reshape(xx.shape[0], 1))\n",
    "plt.plot(xx, regressor_quadratic.predict(xx_quadratic), 'r-')\n",
    "\n",
    "cubic_featurizer = PolynomialFeatures(degree=3)\n",
    "X_train_cubic = cubic_featurizer.fit_transform(X_train)\n",
    "X_test_cubic = cubic_featurizer.transform(X_test)\n",
    "regressor_cubic = LinearRegression()\n",
    "regressor_cubic.fit(X_train_cubic, y_train)\n",
    "xx_cubic = cubic_featurizer.transform(xx.reshape(xx.shape[0], 1))\n",
    "plt.plot(xx, regressor_cubic.predict(xx_cubic))\n",
    "plt.show()\n",
    "print(X_train_cubic)\n",
    "print(X_test_cubic)\n",
    "print('二次回归 r-squared', regressor_quadratic.score(X_test_quadratic, y_test))\n",
    "print('三次回归 r-squared', regressor_cubic.score(X_test_cubic, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "从图中可以看出，三次回归经过的点更多，但是R方值却没有二次回归高。下面我们再看一个更高阶的，七次回归，效果如下图所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEaCAYAAADzDTuZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe4HGX5xvHvTSiBBAgdhEBAUYr0ThACKgnSQenSREFC\nR6qKoIiI9A5SAhEQEJCi8AMhCNJ7FUQFIQKRkkBCSUJy//5457Cbk9O3zOzu87muubI7uzvznDd7\n5jnzVtkmhBBCa5sl7wBCCCHkL5JBCCGESAYhhBAiGYQQQiCSQQghBCIZhBBCIJJBCDOQVLffCUkn\nSJqzD5+btRbxhNYWX6pQCJLmBu4Huhr4siLwQPZ4NeCpDh6/bHs/Sf2Ba21vnR1/f+D7wCa2x5ed\ndzegn+0rsgvzI5JOBKZlb/mb7XHZe5+wvUYXP8PSwBfKdm0PTATuLNv3nu2XJO0KLAx8VdIpZa8v\nACxmeyFJywBrAwOACcBg4EbgOEln2X6u7NwnArcCmwFbZOdtsxhwlO2bO4s9hEgGoSj6AeNtbyxp\nY2BT28dIesz2Wu3fLOlI26dkj8+1fUC7t2wL3JO9PgJYDtgZuFLSrrY/lHQ+sEl6i75Auug+D/wN\nmBe4BhieHaMfsKykhzqJ/xngImCJthCzY8xWtq8t9hHA/sAWWWLaONu/AOmC/p3sra+SEsp4YG6g\nv+3XJa0HjJG0MilJzAcMJSWS/sC+tp8oO9/3Ook5hM9FMghFNAcwtf1OScsCB5PuHp4pe2l+SUvY\nHlu2b3/gp5J+DCwOHGB7uqSfAbdKusT2/pJ+DryZHW/17LPfIf2FfVjbXUF2jPtsb9FV4JJWB/YB\nJpGSi0mJaC7gGtvXSvoNsBVwoaTjbL8saX3gKuBq2/dmhzsG+Cbpzmcs8I6kCcDL2fEPAEYDf89+\nhr8DXwLOk1R+Z7AA8OOu4g4hkkEoonmYsZqjzeKkv5DHAl+WdC6wLvA+cLWkeYBTgc+AZYCBpAvk\nW8BKpAv+d4G2OwWArUkX7N8A5wO7Ar8C/gtMkSSnOVsWAtaV9FgnMQ+1PSU75xjgYdLdQZuVSXcK\n2D5C0nDgQ2CApKuAOUnVOwdJuo2U9E4m/Y7Olv17MnA0MM72zZIOAv6Y3eX8G3iEdJcw0vYTko4C\n7rBdnjhD6FAkg1BEywHPlu/IqlZWAl4Evk66aE8ALgX2BY4Hxti+Q9LpwOWkaqcHsvr0f2SHWhWY\nbPspSZsAHwCXAMsCdwO3k/66Xgr4EXAosH1W7bJgR8FmbQlTynZNo9TmACkpTMv+RdJgYBSwG7A3\ncK7ttuqnfSUNA75HSmqPAO8C/wI2Ag4DzsoakefKEsGSwArANqTkt7mkj4AhwHeyu4SXbe/XUfwh\nQCSDUDCSFibV95+a7Zqa1aWvSqo7H0RKAHuTqlAM3JK9d3VJXwMOB44rO+x8pIt+e3sD15Iu0heQ\nGnS3yrY22/f2RwC2JN2xlJsfuCF7/DvgIdLFfgxwkiRIDb1vZe85Etgl+zmWBl4AJto+VtKLwGnZ\nMQCOJSWqg0lVXIfZHptVid1W3n4QQmciGYSiMKmr82jgYNsfZ/vPBW4j/XU8ilRVNI50wZ5ue9OO\nDpZdXNt6Jq1C6c6g3BGkBuRZbY+TtLztr5Ud43JgTkk3knrydBTzCcxYHWTSRf+BdvtXI7WFAHyb\n1KaB7Ruy9yPpdtublZ3/JVIPq6Wz8x+VvXQ+8DTwlewY+0n6BfBR9r59JX2TVA21QXbn8APbf+3g\nZwgBiGQQikPAdGAz29Pbdtq+GrgaIKs+mZ10Ab+RUq+bTo+ZfeZN25Pbv2j7rSxpdMn2dl2c4CvA\n2+1+jgGkuxGRkoNIbR1TsuO908PzTpR0DKn6aiFg02wcxFGkNolDJY3M2jRmIbVJfEq64xjZdkcg\n6fBuTxZaXs2TQVY/eiXpFtzAxbbPlnQ8qdfFO9lbj7F9R63jCYX1GakK6IkuLpT9SW0BZ5C6YO7T\ng+MeBYwsez6FmccyfP5c0v1l+78E/Kz9ASWdAOwITCb1Ejog29/WVXUKqatnuYGkxuKVbe9Ydqwx\nZe8ZUPb8VVL7xS7Ag9l2MPBl4EHbu0q6APiDpP+QqswEnE3qqlreo2gw0FnDdwgAqNaL20haFFjU\n9tOSBgJPkBq6diDVgZ5e0wBCaFCS1gJesj2xbN/ath8te76k7dezx209n0LotZrfGdh+m+w22vYk\nSX8n1fvCjHWqIYQytmf6a748EWTPXy97HIkg9Fld5yaSNITUkPZwtutASc9IulTSoHrGEkIIoaSe\nk3INBP5A6ikyidSVb2lSl8G3SF3lQggh5KDmbQYAkmYjdQ+83faZHbw+BLjV9krt9sdtbwgh9IHt\nXlXD16M3kUiDhF4sTwSSFrPdNsBmW+C5jj7f2x+oWUk63vbxecdRBFEWJY1aFhL9SN1g57T5rDrH\nbMyyqIW+/CFdj3EGQ0nD7p+V1DbN8LHAzpJWJXXre5U0pUDo3JC8AyiQIXkHUCBD8g6gjxYExlcr\nEWSGVPFYLacevYn+RsdtE7fX+twhhMJalDSSPBRErHTWOEblHUCBjMo7gAIZlXcAfbQI1U8Go6p8\nvJZSlwbkvsrG0ESbQQhNRmJ3YFOb3fKOpRn15doZdwYNIptjJxBlUa6By6LqdwYNXBaFEMkghJCH\nRZhxgr+Qs6gmCiHUncTvgDttrsw7lmYU1UQhhEZRiwbkUIFIBg0i6kNLoixKGrgsFqXK1UQNXBaF\nEMkghJCHuDMomGgzCCHUlcSswCdAf5tpecfTjKLNIITQCBYC3o9EUCyRDBpE1IeWRFmUNGhZ1KSK\nqEHLojAiGYQQ6q3qjcehctFmEEKoK4k9gG/YfDfvWJpVtBmEEBpBjD4uoEgGDSLqQ0uiLEoatCxq\nMn11g5ZFYUQyCCHUW4wxKKBoMwgh1JXE3cDJNnflHUuzijaDEEIjiDuDAopk0CCiPrQkyqKkQcti\nMDC22gdt0LIojEgGIYS6kZiXdN0Zn3csYUbRZhBCqBuJlYFrbFbMO5ZmFm0GIYSiWwr4T95BhJlF\nMmgQUR9aEmVR0oBlsSTwei0O3IBlUSiRDEII9RR3BgUVbQYhhLqRuBa42ebqvGNpZtFmEEIouppV\nE4XKRDJoEFEfWhJlUdKAZVGzaqIGLItCiWQQQqgLiTmABYE3844lzCzaDEIIdSHxReBumyF5x9Ls\nos0ghFBk0ZOowCIZNIioDy2JsihpsLKoaTJosLIonEgGIYR6iZ5EBRZtBiGEupC4DHjI5rd5x9Ls\nos0ghFBk0WZQYJEMGkTUh5ZEWZQ0WFnUtJqowcqicCIZhBBqTmIW0qI20WZQUDVvM5A0GLgSWBgw\ncLHtsyXND1xLunV8DdjB9oR2n402gxCagMRiwDM2C+cdSysoapvBVOBQ2ysC6wIjJS0PHA3cZfvL\nwN3Z8xBCc4qeRAVX82Rg+23bT2ePJwF/BxYHtgKuyN52BbBNrWNpZFEfWhJlUdJAZVHzxuMGKotC\nqmubgaQhwGrAI8AitsdlL40DFqlnLCGEuoqeRAU3a71OJGkgcANwsO2JUqk6y7Ylddh4IWkUqU0B\nYALwtO17s9eGZZ9v+ue27y1SPPG8OM/bFCWeTp4vCWdJOmRYrc7Xtq8gP29dn2eP98yK4TX6oC6D\nziTNBtwG3G77zGzfS8Aw229LWgwYY3u5dp+LBuQQmoDELcDlNjflHUsrKGQDstItwKXAi22JIHML\nsEf2eA/gj7WOpZFFfWhJlEVJA5VFtBkUXD2qiYYCuwHPSnoq23cMcDJwnaTvkXUtrUMsIYR8RG+i\ngou5iUIINSUxD/AWMNCmuBecJlLIaqIQQstbCvhPJIJii2TQIKI+tCTKoqRByqIuVUQNUhaFFckg\nhFBrXwRezTuI0LVoMwgh1JTEBcCLNufkHUuriDaDEEIRrQC8mHcQoWuRDBpE1IeWRFmUNEhZLE+a\nk6ymGqQsCiuSQQihZiQWBGYndS0NBRZtBiGEmpH4GnCKzXp5x9JKos0ghFA00V7QICIZNIioDy2J\nsihpgLKoS3sBNERZFFrdprAOITQJqR+wIGkp2/mAeYFBwECgPzAnMAfQb3le2OoHXPwoOusXwBRg\ncvbvRODDbJsAvJNtkyhy3XUTi2TQIMrnbG91URYlNSkLaXZgGeDLwLKk6SSWzLbFgflJF/D/Ae8D\nH2TPJwGfAJ+SLvqT/8viC6zBE2NJy9/OAcxNShgDs8fzkBLKQqTkMgvSm8AbwFjSTKf/BP6V/ftW\nZ8kivheViQbkEFpVml5+KWANYKWybUnSxfgfpAvwa6SL8uvAf4F3sD/r/vDMA5Pfg7kegOkfA7vY\nntDNhwaQEs4SwOAsvi8CXyIlptlJbRDPA88BTwJPY3/Umx+92fXl2hnJoEGUr+DU6opaFhJzAisD\nq2TbUsDHpL+YJwJPAXfYvF29c/aiLNJqg+sCGwDrAGsCnwGPA8+QLq7PAa9gT608NtaGl+6G5Qdm\nu66zvWOFB10QWDHbVgZWzx7/52p4bRe4EXgQeAl7ekXnamB9uXZGNVEIFZJYBfghsCNpDp5nsu3/\nSPXnA0n16psDZ0j8C7gJONfmgxoGNgD4GvB1YGNgOVJC+htwIfAY9ps1Oz+sAK9+SPr5HwP2rfiI\n9rvAX7MtSSsprvgW7A5sRFovZRDSX4Ex2fZCtEV0Le4MQugjiU2AX5KqNC4GLrXp8uIqMRuwHrAX\nKTn8hpQUPqlCQCJ15fxWtq0FPAHcDdxDuvhPrvg8PQ6HX8P4KTD/l4F9u60iqu7JFweGkZLgJqT2\nijuA24G/UM9YchDVRCHUgcSiwGmkVfx+BPzRpts69A6OszxwIqnq5oA+rQ+cevYMBbYDtiZ1F/8T\n8GdgTJ516YVa91haFhgBbEaqJnuEtNTuzdhj8wytFmLQWROLPtQleZWFhCT2IdWrvwGsaPOHviQC\nAJu/22wPfAfeuUIa9YY05+2SBnUTyCxIw5AuugveBc4C3gO2AoZg7499WwEaVVegTmMMoJvvhf0K\n9jnY3wIWAy4gtZs8g/QQ0iHZ3UTLimQQQg9I9AcuAQ4CNrY52qYqF1ubB2GlZ2GBJeCeEbDC6E6C\nWAXpNFIiOhP49+mwP/Zq2L/Afq4o9eJZY/ripC6hxWJ/hH0j9u7AosAJwKrAc0j3Iu2NNE++QdZf\nVBOF0A2JxUm9VF4H9rKZVP1z6M+gzeDnb8Cxs8Msw22eyXrP7EJqY5gfuBK4Grtuf3H3RdaofrXN\ninnH0mPSHKS2lt1JbQ1/Ai4jVbc1VM+kqCYKocok1gAeJdUv71CLRJDZBXwd/HRlwYGzM/meOzTi\nduAVYG1S28TS2D8teiLILE+jzUlkT8a+CXtb0riGh4EzgJeRjkRaON8AayuSQYOINoOSepWFxLqk\nhtgDbH5VywXdbU8w7G3YeTr9jr+cvT7enhs2WItHd8DeDfvujv46LfD3oq7tBVDlsrDfxT6HNF5k\nN1K33JeRfoe0dtXOUyCRDELogMQGwC2kaqHa9oaRlkQ6hTTKd1PggF24ZsmPGbDT46x1VZaUGk3j\n3Rl0xDb2I9h7k0ZCPw1ci/QI0k5ITTNWK9oMQmhHYhhwHbCrzV01PNHKwBGkeuorgHOx/90uls1J\nDddDbf4980GKSeIFYBebZ/KOpepSd94tgMNIo8zPAC7FrlUVYq/FOIMQKiSxJmlg0g42Y2p0kqHA\nj0k9WM4CLupqEJTE/qReTOvZjK9JTFWU5iTiTWB+myl5x1NT0jqk9pxhwLnA2di5/x9FA3ITK3Dd\ncN3VqiwkvkiqGtqnJolA2gjpbmA0qUF6Gexfdzca1uZ8UtvFjRKzz3jIQn4v1gGerHciyKUsUhXS\nd0gD/5YC/ol0EtJCdY+lQpEMQgAkFiZNV3CCzc1VPvgGSPeSqntGA1/Bvhj7014c5QjSVNEXSRT9\nbnko8EDeQdSV/Y+sXWEN0pTcLyGdiDRfzpH1WFQThZYnMYA0mdn/2fy0igdenTTdxAqkgU2jezL1\nc+eHYwBpRs7zbS6qTpDVJ3EXcLbNrXnHkhtpCPATYBtSVeAZ9WxTiDaDEHop+yv7OtJU03tWpfuo\ntAxwErAhaSK7S6o1QZzEl0l/dQ+3ebIax6wmiVlJC94sbfNe3vHkTvoS8HNSm8IvSN+FiqcH7/60\n0WbQtApaN5yLKpfF0aTFXPatOBFICyCdQZqu+QVgWezzqjlTqM0/gAOA6yUGFfB78VXgv3kkggKW\nBdj/xN6F1PtoO+B5pG2yGWYLJZJBaFkSm5EurNvZ9Kb+vv2BZkM6iDTIag5ghWyuoJpMFGdzLanH\n02UF/BVuvfaCnrCfxP4mcCCp6vBupFVyjmoGhfsmhY4VcWWvvFSjLCSWJfXt38HmvxUcaDhpIZst\ngI2zGUPHVRpfDxwODIZpK9fhXL2RWzJoiN8R+05Sl+LrgTuRLi5Kz6NIBqHlSMxFWmnsOLuPFy5p\nKaSbgPNIVU3DsV+oXpRds5kM7Az8VOKr9TpvD8SdQXfsz7AvAL5Caqt6AWm/bDBbbiIZNIhC1ofm\npAplcQ5p+cfe98iRZkc6mrSC2JPAV7FvyWPqaJt/wq8vB66SmKPe529PYglgLtLkejmcv8F+R+wJ\n2IcA3wB2BR7Nc96jSAahpUjsRlrpav9eNxhLG5DmptkAWDtrF+h7W0NVHPtn0poBJ+YbB5DuCh6s\n5YR+Tcl+ltTz7EzgZqSz81hPoS7JQNJlksZJeq5s3/GSxkp6KttG1COWRtUQ9aF10teyyLplnkFq\nJ5jYiw8OQroQ+D2p7/iW7ecQyos97V7gB8DO2ZrMeVqfHKuIGvp3JE2INxpYERhAqjratp4h1OvO\n4HLS+qPlDJxue7Vsu6NOsYQWlK1Udh3w015NniZtDTyfPftqtkJWof7ytXkX2BsYJdH1kpm1Fe0F\nlbLfx/4eadrsk5GuR1qkHqfuNhlI6idpd0l/lvT3bHte0uWSejS1ru37ocMJtgrX17aoGq4+tIb6\nWBa/JtVl96ydII0ZuAo4FdgFe7/u5hDKQ1tZ2NxJmlfp7HziYCBp2uon8jh/iqGJfkfsv5J6Hf0b\neBZpl1qPTegyGShlpD8BSwAjbS9ve3nSgg8XAj+QdH4F5z9Q0jOSLu12EfAQ+khiOLAt8IMe1Wen\n2/PngHHAKtj31TbCqjkKWFdiuxzOvTbwTEXjNcKM7E+wjyJ1Wz4WuKmWq611OR2F0rD6T2y/1cV7\nNnLKYl2fKM3VcavtlbLnCwPvZC//AljM6fao/DMxHUWoiMSCpHEA37W5p5s3z0v6y3oosCf232of\nYXVJrEfqNruKTT3GO7Sd9xRgqs2P63XOlpLWZz6etBb2/tg3dv32Plw7U7tF7TdgCPBcb14jtSuM\nygrheOAQYFjZ68PieTzv/Pksw+D6+8Gndvt+GHYHvD0abjEMLEb8fXsO/hX4j+nnr8v5ZgG/Djvs\nVYSfv5mf7wsjDa8YrvwybN72evbeUZSul277XE+3Xr25ko12F3zSnUDb40OBqzv4TK9/oGbdyr8U\nrb71tCzAe4OfBc/R6ftgdsOvDW8aNs/7Z6tGWYDnAD8D3rM+Mfhr4A7/0Mu7LJpygwGGCw2vGtbv\npCzc2+PWZf1OSdcAGwELSnoD+BkwTNKqpL/+XwX2rUcsoTVIDCE1Gm/iNFq3ozctC1wDvEVqG3in\nw/c1GJvJ2XiKeyTuc+2Xy9wFuLrG5wht0pxX+yFtQ2pHuAA4kQqmR4deTGEt6TzbI7PHpwBrAZfY\nvqqSALo5px1tBqGXJGYB7gZutzmlkzftQeopdAJwHj39RWggEocC3wY2sqnoQtHFOWYjLXG5ts2r\ntThH6IL0BdIcW/1Jvd7eSLurPIW1pN+VPV297YHtI0n9mo/rzclCqJMDSLOHnjbTK9IApFGk+YS+\njn1uMyaCzFnAJ8AxNTzHN4FXIhHkxH4TGE7q9fkY0uZ9PVR34wy+2HkMfhV4t68nDr3TVH2oK9RV\nWWSjjI8jLVQzrd2LK5LWGpgFWIs0DUBD66osbKYDewIHSNRqzpvCVBG17O+IPR37ZNJd4AVIp/bl\nML1pM+gv6fp2nx3Ql5OGUAsS/Ui9KX7utAhM+YvfBU4HjgRGNfHdwAxsxkqMJE1mt4bNh9U6djb7\n6xak6bRD3uy/Ia1Gqjbqte7GGTxke73s8cPAHpRGDRt43fYnfTlxj4KLNoPQCxJHAN8Cvp79VdzW\nP/sM0syQ22M/1/kRmpfEhcDCwPapQ0pVjrkjsLfN8GocL1SJNItgWm+vnb25M7Dtl3sZVgh1IbEc\naQTuWmWJYDDwB+C/pGqhD/KLMHcHA/cDR0Anjeq9V5gqolDGnk4fZq7ozUR1c2VzEj0o6cxsRHGo\nk5atD+1A+7LIqocuB372eUOmtCHwKHAD6Y6gKRNBT78XWffabwOHVWN2U4mV4fPRzoUQvyOV6U0y\n+Nj2V4FtgGeBOyWt3s1nQqiHQ4DJpMYzIY0kzVD6XexTWqV9oDs2r5Nmw7wqW4imT7KuuxcAP6lm\nG0TIV3dtBuVjCz5vP8ier0y6RVzd9pSaBBdtBqEbEl8hTZu8jtFY4HzSpGlbU5A1B4oma1vZA9jY\nptcD7ST2Jg0SXe/zKrlQKH25dnbZZtCWCDJ/bffas9mMpQuSBp2EUFdZ9dBlwAlGE0kDzcYB62FP\nyjW4YjsVGATcJbGJzfs9/aDEAsCvgM0iETSXnqxn8CNJswLHSTo32/eL7OWxwGI1jC9koj60pKws\nDgSmTWLAX4FHgDHAd1opEfTle5H1JvoJ8Bfgzl4uiPMr4FqbJ3t73lqL35HK9KQ30eakBRamkZZk\ng7QGLMDGpAU1QqgriS8BP7mC3Y8ZwMd/AQ7Bjp4tPWTjrLroLOD/JHa0ea2rz0h8k3Q9WKEOIYY6\n6/TOQNJykrZoe0q7VcmUVt1ZG2i4Od8bkRt5fdeq833ApTtxzX27M/oEUvtASyaCSr4X2R3CwaQe\nQY9LHCbN/AeiRH+J3wBXktaFKGTPrPgdqUxXdwZvkHoMzEYaYNa+pfkQ4ArbU2sUWwgdmpWpP1yK\n/yw7mu8uBAwlTY0S+iBLCCdL3EBaEnRXiTNJC0+9C8wDnEda+W3lvjQ4h8bQ6Z2B0zSpmwHzZv9+\nC1hI0k7AasACti+uS5Qh6kMzc2u1a2fnT+ecxNZznsq0zVo9EVTre2HzCvB1UrXRt4DDSMnhbNL4\njR2Kngjid6Qy3fUm+lTSdOBT0p3BdODj7HNLSJrFdvQoCHUxSQMXWp5bt5+Li7UbLw6aCqccDTvm\nHVezyO4Srsy20GK6ajOYU9LNwHjgXuAe4D3bt5BmfnyQtHZxqIOWrw+Vlr6EfZ5/m0Gf/ZXrmZq+\ngy2/IFLLfy/KRFlUpquupV8lTfBlOmgzyKqINlRaRDyE2pFW/xfLPHQsJ831BhM3gmnXAZvanpB3\naCE0i67aDB4ry7TTs62920iLW4Qaa9n6UGnj6eiODblv3CfM9Ut7w0eACyIRJC37vehAlEVlejI3\n0bW2/2j7VlKPAoA7sn/vhZibJNSItB1w7QjuuOJNFp9CGjkbQqiBHq+BnIeYm6iFSd8DfnE+P9xn\nJOdfQZpH5/m8wwqhEfTl2hnJIBSPdDhw4EQGDp+HiecDd9mcnHdYITSKvlw7ezOFdchRS9SHpumn\nTwC+D2wwDxO/QVpa9dR2bxuWQ3SFFGVREmVRmS6TgaQvSvpCN+/ZoKvXQ+iRNL3JaaT1MjYU7g+c\nAOxh81musYXQArpbz2BR0gLj9wNXu2y0p6Q1gf2Az2zvV5PgopqoNUhti6WsCmwm/AGpc8KNNmfk\nGVoIjagmbQbZ9NU7ZdsypAnrpgNPAhfZrtlEdZEMWoDUD7gUWBrYAnuixOHA1sCwmDM/hN6LBuQm\nJmlY042wTH9ojCKtibEV9kcSK5FGu6/9+XrGM32sCcuij6IsSqIsSqq+0lkINSPNBowG5ge2xP5Y\nYg7gKuDIzhJBCKE24s4g1F9KBFcBA4HtsD9NuzkF+BKwfTZpWgihD+LOIBRfqhr6HTMngo2A3YBV\nIhGEUH/ddS1dRNIi2WNJukHSwpKWyrYl6xNmaIo+1CkRjCatkVGeCAYBVwDf78mc+U1RFlUSZVES\nZVGZ7u4MNgB+KukwYArwHnAD8M/s9RGkxr8QupZ6DV1BaiPYuiwRCLgYuNXmTzlGGEJL626cwfbA\ni6SprAcBuwOX2v5a9vr9bY9rEly0GTSHNI7gUmAwqbH4k9JL7AMcROo99GlOEYbQVGoxHUU/4APg\ndmBx4D99jC20qjSy+FxSw/DW7RLB8sDJwE6RCELIV1crnc0C/IQ0L8xCwDGkv+BCDhqyPrQ0xcQa\nwOakdbWzl+gP/B44xubFXh52WDXDbGRRFiVRFpXpanGb6cA6wL+Bt0hdATcDnpM0RtIYiDljQpd+\nDmwMjMBuv+7FacArwCV1jyqEMJOeTEdxDGmemGWAZUmrTI2rfWjRZtDQpB8B3wM2xH5nxpfYmbR+\n9po2sWJZCFVW9XEGkvYA9gTGkyalOx0Yke7+AbDtK3sQ2GXA5sD/bK+U7ZsfuBZYCngN2CGWMmwS\n0veBkcDXOkgEywNnA9+MRBBCcXTXgDwNMPAF0gR1+5AGC32WbdN6eJ7LSd1Qyx0N3GX7y8Dd2fPQ\niYapD5V2BI4HNsUeO+NLDCR1TT7K5um+n6JByqIOoixKoiwq0+Wdge3fSVof+AfpDmE+YG/gCNtv\n9fQktu+XNKTd7q2AjbLHV5CqoiIhNDJpONlf/divzPgSAi4EHra5LI/wQgid60mbwTzAraS/7Kdk\nu9s+ZPdwcqMsGdxaVk003vZ82WMB77c9L/tMtBk0Cmkd4DZgG+wHZn6ZQ0njVIbafFzv8EJoJbVo\nM5gPuBI4FDiJtPhIuSnA8N6csCO2LSnmo2lU0vLAzcCenSSCTYEjgXUiEYRQTN1NR3EPsLPtl0iL\n2VTTOEmL2n5b0mLA/zp6k6RRpAZmgAnA021zlrfVEbbC8/L60CLE0/Z8GCw0JnUTPVLwEWVzyqf3\nb7k43HIg52fmAAARL0lEQVQ68G3QMhLLVHr+9mVSpPLI4fmqts8sUDx5Pj+E1r4+7EnyGn3Q3XQU\n+wPLAYcAKwFzk0Yk/8P25F6daOZqolOA92z/WtLRwCDbR7f7TFQTZVTEhTvSnePfgMuxT535ZeYB\nHgbOsrmoeqctYFnkJMqiJMqipC/Xzp60GYwgNfY+ThprMB+wCvAmcLTtf/cgsGtIjcULAuOA40jV\nCtcBS9JJ19JIBgUm9QfuBB7HPmzml5kNuAV41Wb/eocXQiurSTLIDvxj4O+2byzbNww4yvZmvQ20\nx8FFMiimNAPp70lrYe9MGq1e9jICfkua0XZrO0aqh1BPfbl2dreewc2SDgYeAKZIGppt6wNTgRP7\nHm7ojcL0oU49v04n3eXt3j4RZH5C6mywYy0SQWHKogCiLEqiLCrTXQPyjsC2pKkDliENHpva7j0z\n9R4JTe0QYBPS6OKZ2o0k9iCNRVnPZlK9gwsh9E2P10CWtBOwNnB4T8cWVCqqiQpG2o40qGx97Ndn\nfpktSdVDw2xeqnd4IYSk6tVE5Wz/3vZh9UoEoWDSoLILga06TgR7/QnG3wDf/Bfo7foHGEKoRI+T\nQchXrvWh0jLATcBe2DONN5HYEE7/Jmw5G/xlfaheN9KOw4m64TZRFiVRFpWJZBC6Jg0C/gSciD3T\nGsUS6wI3wA+eypqPHgP2rW+QIYRK9bjNIA/RZpAzaTbSkqfPYx8y88sMI40V2RP0IOmOYN/240VC\nCPVVs3EGeYlkkKPUhfRCYAlSO8G0GV/mW8AoUvfRMfUPMITQmZo2IId85VAfeiiwHrBTB4lgB1I3\n463ySARRN1wSZVESZVGZ7sYZhFYkbQH8CFgPe2JpNwIOBo4grVT2bE4RhhCqLKqJwoykFYExwNbY\nD5V2MxtwDrA+sKXNf3KKMITQjb5cO+POIJRIC5Aml/tRu0QwCLietH7FBjYf5hRhCKFGos2gQdS8\nPjT1HPoD8AfsK0u7WYM0Y+0LpDaC3BNB1A2XRFmURFlUJpJBaHMW8BFwLKT2AYmDgDuAY20OsZnW\n1QFCCI0r2gwCSN8HDgPWwf5QYiHgYmAwqevov3KNL4TQK9G1NPSeNBT4JbC18ESJ3YDngX+SFq+P\nRBBCC4hk0CBqUh8qLU42glh4MmnaiSOALWyOsOnV0qb1EnXDJVEWJVEWlYlk0KrSspU3vsZSlwhv\nCDxJmlxoTZvH8g0uhFBv0WbQiiR9wDyXn8Sxq5zCkV8A3QYcZ/PfvEMLIVQuxhmEbknMvzm3XvYI\n62z+AfPeCdrd5rm84woh5CuqiRpEJfWhWTfRtSXOn5Wpr87H+OE7c81WUzz75o2YCKJuuCTKoiTK\nojJxZ9Ck0jxC6/8etlkLdlwYlhi3MO9e9zhrThrM2B9i3553jCGE4ih8mwH4ri7e4nbb9Gwz8Fm2\nTQOmlm1TgE+z7ZNs+5g04OpjYGK2fZhtE4BPbIpbUBmJ+YF1geHA5vC/wXDj7DCaWXno+ql4YeA+\n7OPyjTSEUEtNuZ4BeHhnL3eyzZJt/Uh3PrMCs5VtswNzAP2zbS5gQPbvwGybB5gbmDfb+pGSwnjg\n/bJ/3yvb2p6/W/Z8Ui2SiMSswBBguWxbiZQEFiNNHXE38CeY5STwZsBjH8CD88DywLfaT0kdQmgu\nTZkMitCbSGIOYBAwX7bNDyxQ9m9H23ykZDOelEgmAB+Q7jYmke5EPgImU7prmUZKPG3JbAApKc0N\nN34RtpuDdMFfEBgLvJRtLwKPAC+UTxmhtGTlRa/BLUvBScAa2O9WvYDqTNIw2/fmHUcRRFmURFmU\nRG+iGskGX43Lth7Lksh8pEQyb9k2oGybg3TH0p+UAKaRqrqmke4wXgMmwmNLwHZ3AW8B/7OZ2n3c\nnoD0Y+BBYItmSAQhhNqIO4NmJs0JPAxcjH1e3uGEEOojqonCjKRLSG0hu1Lk/+gQQlXFRHVNrNd9\nqKXdgQ2AfZstEUR/8pIoi5Ioi8pEm0EzklYATgM2KV/DOIQQOhPVRM1GGgA8CpyKfXne4YQQ6i/a\nDAJIowCw98w1jhBCbqLNoIn1qD40tROsDYysdTx5irrhkiiLkiiLykSbQbOQvkKpneCjvMMJITSW\nqCZqBqXxBOdhX5x3OCGEfEWbQauSzidNgbFTs3UjDSH0XkO2GUh6TdKzkp6S9Gje8RRVp/Wh0nak\nWUp/0CqJIOqGS6IsSqIsKlOENgMDw2y/n3cgDUcaDFwAbIX9Qd7hhBAaV+7VRJJeBda0/V4Hr0U1\nUWekfsA9wO3YJ+cdTgihOBqymoh0Z/AXSY9L+n7ewTSQY0mL95ySdyAhhMZXhGqiobbfkrQQcJek\nl2zf3/ai0iCq17KnE4Cn2+Ysb6sjbIXn5fWhTmsfjNwARj4AGxpyj6+ez9uXSd7x5Px8VdtnFiie\nPJ8fQmtfH/YkeY0+yL2aqJyknwGTbJ+WPY9qosznC3dI8wJPAwdj35J3XHmIRUxKoixKoixKGq5r\nqaS5gH62JyrNqXMncILtO7PXIxm0J40GJmH/MO9QQgjF1IgrnS0C3CSpLZar2hJB6IC0M7AWsHre\noYQQmkuhqonaizuDkm9IO/4FzgFGYD+Zdzx5iuqAkiiLkiiLkkbtTRS6I/U7KPUeOrXVE0EIoTbi\nzqARSEcBI4BvYE/LO5wQQrE1YptB6I60KnA4sGYkghBCrUQ1UZFJ/YHfAYcJlsk7nKKIOWhKoixK\noiwqE8mg2H4JvAhclXcgIYTmFm0GRSVtAlwJrEIH8zaFEEJnojdRs0ijjC8H9olEEEKoh0gGxXQm\naTbSO9p2RH1oSZRFSZRFSZRFZaI3UdFIWwEbAqvkHUoIoXVEm0GRSAsCz5KWr7wv73BCCI2p4Saq\n605LJYM0QdO1wBvYh+cdTgihcUUDcmPbAVgJ+ElHL0Z9aEmURUmURUmURWWizaAIpEWAs0hrGX+S\ndzghhNYT1UR5S9VDfwBewT4673BCCI0v5iZqTDsCywG75h1ICKF1RZtBnkrVQ3thf9r1W6M+tE2U\nRUmURUmURWUiGeQlVQ9dAFyG/Wje4YQQWlu0GeRF2gE4Hli9u7uCEELojWgzaBTSQsDZwDaRCEII\nRRDVRPk4C7gK++GefiDqQ0uiLEqiLEqiLCoTdwb1luYeWhtYOe9QQgihTbQZ1JM0CHge2BX7r3mH\nE0JoTjE3UdFJvwU+w/5h3qGEEJpXzE1UZNLGwAjgqL59POpD20RZlERZlERZVCaSQT1IcwK/BfbH\n/jDvcEIIob2oJqoH6dfAUtg75R1KCKH5xTiDIpJWB/YkTU8dQgiFFNVEtSTNClwCHIH9v8oOFfWh\nbaIsSqIsSqIsKhPJoLYOBt4DRucdSAghdCXaDGpFGgI8DqyD/a98gwkhtJLoWloUpRlJT41EEEJo\nBJEMamNHYHHgtGodMOpDS6IsSqIsSqIsKhO9iapNmh84A9gWe2re4YQQQk9Em0G1SRcDU7APyDuU\nEEJrinEGeZOGApsDK+QdSggh9EaubQaSRkh6SdIrkvo0Z09hSLMBFwKHYn9Q/cNHfWibKIuSKIuS\nKIvK5JYMJPUDziVN3rYCsLOk5fOKpwoOA8YC19fo+KvW6LiNKMqiJMqiJMqiAnlWE60N/NP2awCS\nfg9sDfw9x5j6RloaOAJYm9o1wgyq0XEbUZRFSZRFSZRFBfKsJloceKPs+dhsX2NJYwrOAU7H/nfe\n4YQQQl/keWdQ3G5MvbM1sAywXY3PM6TGx28kQ/IOoECG5B1AgQzJO4BGlmcy+C8wuOz5YNLdwQwk\nNUrSmIxq2wtW0h41PUEDibIoibIoibLou9zGGSjN6Pky8HXgTeBRYGfbjddmEEIIDS63OwPbn0k6\nAPg/oB9waSSCEELIR6FHIIcQQqiPwk5U11QD0iok6TVJz0p6StKjecdTT5IukzRO0nNl++aXdJek\nf0i6U1JLdCnspCyOlzQ2+248JWlEnjHWi6TBksZIekHS85IOyva33Heji7Lo1XejkHcG2YC0l4Fv\nkBqaH6OF2xMkvQqsYfv9vGOpN0lfAyYBV9peKdt3CvCu7VOyPxTms310nnHWQydl8TNgou3Tcw2u\nziQtCixq+2lJA4EngG2AvWix70YXZbEDvfhuFPXO4PMBaU4zf7YNSGtljTVhX5XYvh8Y3273VsAV\n2eMrSF/8ptdJWUALfjdsv2376ezxJNJg1cVpwe9GF2UBvfhuFDUZNMeAtOox8BdJj0v6ft7BFMAi\ntsdlj8cBi+QZTAEcKOkZSZe2QrVIe0qrCq4GPEKLfzfKyuLhbFePvxtFTQbFq7vK11DbqwGbASOz\n6oIAONVztvL35QJgadK8PG9RxQWVGkFWLXIDcLDtieWvtdp3IyuLP5DKYhK9/G4UNRn0aEBaq7D9\nVvbvO8BNpGq0VjYuqydF0mLA/3KOJze2/+cMcAkt9N1Qmin4BmC07T9mu1vyu1FWFr9rK4vefjeK\nmgweB5aVNETS7KRlJG/JOaZcSJpL0tzZ4wHApsBzXX+q6d0CtI003QP4YxfvbWrZBa/NtrTId0Np\nTrBLgRdtn1n2Ust9Nzori95+NwrZmwhA0mbAmZQGpP0q55ByoTQj6k3Z01mBq1qpLCRdA2wELEiq\nAz4OuBm4DlgSeA3YwfaEvGKslw7K4mfAMFI1gIFXgX3L6syblqQNgPuAZylVBR1Dmsmgpb4bnZTF\nscDO9OK7UdhkEEIIoX6KWk0UQgihjiIZhBBCiGQQQgghkkEIIQQiGYQQQiCSQQghBCIZhBYl6QRJ\nc+YdRwhFkecayCFUVTZA7wtlu7YHJgJ3lu17D1gDWBj4ajYddpsFgMVsL9TJ8RcFBtj+VwevfQn4\nie09s+enAtfbfqSbmL9Nmp78mA5eWwb4qBUGkYX8RTIIzWQQsET2WMC8wGxl+wA2BkYAW9oenz1H\n0gLArcB3Ojpwtmb3KGDvdvtPBoYC/YHBku7PXhoCfEPSROAl0ojh4R0cej5gLknfAB63/cOy1z4B\nrpC0ue1p3f3wIVQiRiCHpiLpe8A+pEVgBpCG4n8MzEVaF2Mw8GvgfOA42y9LWh+4Crja9o87Oe5u\nwFK2f9nJ6wsD25Hmkp8fmBN4pP1dhKTZbU8pe7496c7g2E6Oeyww1vaVPSyCEPok7gxCsxkIjCHN\n516+sMfKwDy2j5Q0HPgQGCDpKtKFewvgIEm3kaYAbl8VtBNwYEcnlHQisDkwgVT9NCvwJnCopIdt\nH5i9b1PSFOTb2p5efogufp5rgPOASAahpqIBOTSjadn2Wba1PUfSYFJ1z+9JVT7n2t7O9gu29yXN\n+f69Do65jO1XuzjnIbY3Bn4FnJE9nmEhItt3khZtuqinP0h2zqV7+v4Q+iruDEKzEbAlsG67/fMD\nNwKjgYdISWIMcFKaAZjFSAuAABzRyXFn3imdQ1pqcStJH5CtrCVpV1I11SKSBtn+bvaRg4HbJK1g\n+8U+/YQh1EAkg9BsTFrk4wFmvICvBswBfBsYCWD7huy9SLrd9mZdHHd6RzuzKqC2aqAlSdU544ED\nbc+0IFPWENzVeWYgqR/ZXU0ItRTJIDQbkf4iny977OzfuYEptt/N7gR660lJa9p+fKYTpkWH9ibN\nH78HMBkYLela4PfdzKe/OKmxuzNrAo/1JeAQeiOSQWgaks4HNgGmkLp7lhtIajBeBXg+e/+YstcH\nlD1/1fbe7T5/EbAfqadS+TmPAnYArgA2sj012z8c2Au4KWtEPqbsM4cDR5J6OU0mVWt1Zl960cYQ\nQl9F19IQekjShaQ1Zv9Wtm8O25NrdL6NgB1t71+L44dQLpJBCCGE6FoaQgghkkEIIQQiGYQQQiCS\nQQghBCIZhBBCIJJBCCEEIhmEEEIA/h+HYNfM90q7egAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x9027c88>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "二次回归 r-squared 0.867544458591\n",
      "七次回归 r-squared 0.487942421984\n"
     ]
    }
   ],
   "source": [
    "plt = runplt()\n",
    "plt.plot(X_train, y_train, 'k.')\n",
    "\n",
    "quadratic_featurizer = PolynomialFeatures(degree=2)\n",
    "X_train_quadratic = quadratic_featurizer.fit_transform(X_train)\n",
    "X_test_quadratic = quadratic_featurizer.transform(X_test)\n",
    "regressor_quadratic = LinearRegression()\n",
    "regressor_quadratic.fit(X_train_quadratic, y_train)\n",
    "xx_quadratic = quadratic_featurizer.transform(xx.reshape(xx.shape[0], 1))\n",
    "plt.plot(xx, regressor_quadratic.predict(xx_quadratic), 'r-')\n",
    "\n",
    "seventh_featurizer = PolynomialFeatures(degree=7)\n",
    "X_train_seventh = seventh_featurizer.fit_transform(X_train)\n",
    "X_test_seventh = seventh_featurizer.transform(X_test)\n",
    "regressor_seventh = LinearRegression()\n",
    "regressor_seventh.fit(X_train_seventh, y_train)\n",
    "xx_seventh = seventh_featurizer.transform(xx.reshape(xx.shape[0], 1))\n",
    "plt.plot(xx, regressor_seventh.predict(xx_seventh))\n",
    "plt.show()\n",
    "print('二次回归 r-squared', regressor_quadratic.score(X_test_quadratic, y_test))\n",
    "print('七次回归 r-squared', regressor_seventh.score(X_test_seventh, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "可以看出，七次拟合的R方值更低，虽然其图形基本经过了所有的点。可以认为这是拟合过度（over-fitting）的情况。这种模型并没有从输入和输出中推导出一般的规律，而是记忆训练集的结果，这样在测试集的测试效果就不好了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "##正则化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "正则化（Regularization）是用来防止拟合过度的一堆方法。正则化向模型中增加信息，经常是一种对抗复杂性的手段。与奥卡姆剃刀原理（Occam's razor）所说的具有最少假设的论点是最好的观点类似。正则化就是用最简单的模型解释数据。\n",
    "\n",
    "scikit-learn提供了一些方法来使线性回归模型正则化。其中之一是岭回归(Ridge Regression，RR，也叫Tikhonov regularization)，通过放弃最小二乘法的无偏性，以损失部分信息、降低精度为代价获得回归系数更为符合实际、更可靠的回归方法。岭回归增加L2范数项（相关系数向量平方和的平方根）来调整成本函数（残差平方和）：\n",
    "\n",
    "$$RSS_{ridge} = \\sum_{i=1}^n (y_i - x^T_i \\beta)^2 + \\lambda \\sum_{j=1}^p \\beta_j^2$$\n",
    "\n",
    "$\\lambda$是调整成本函数的超参数（hyperparameter），不能自动处理，需要手动调整一种参数。$\\lambda$增大，成本函数就变大。\n",
    "\n",
    "scikit-learn也提供了最小收缩和选择算子(Least absolute shrinkage and selection operator, LASSO)，增加L1范数项（相关系数向量平方和的平方根）来调整成本函数（残差平方和）：\n",
    "\n",
    "$$RSS_{lasso} = \\sum_{i=1}^n (y_i - x^T_i \\beta)^2 + \\lambda \\sum_{j=1}^p \\beta_j$$\n",
    "\n",
    "LASSO方法会产生稀疏参数，大多数相关系数会变成0，模型只会保留一小部分特征。而岭回归还是会保留大多数尽可能小的相关系数。当两个变量相关时，LASSO方法会让其中一个变量的相关系数会变成0，而岭回归是将两个系数同时缩小。\n",
    "\n",
    "scikit-learn还提供了弹性网（elastic net）正则化方法，通过线性组合L1和L2兼具LASSO和岭回归的内容。可以认为这两种方法是弹性网正则化的特例。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "##线性回归应用案例"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "前面我们通过一个小例子介绍了线性回归模型。下面我们用一个现实的数据集来应用线性回归算法。假如你去参加聚会，想喝最好的红酒，可以让朋友推荐，不过你觉得他们也不靠谱。作为科学控，你带了pH试纸和一堆测量工具来测酒的理化性质，然后选一个最好的，周围的小伙伴都无语了，你亮瞎了世界。\n",
    "\n",
    "网上有相关的酒数据集可以参考，UCI机器学习项目的酒数据集收集了1599种酒的测试数据。收集完数据自然要用线性回归来研究一下，响应变量是0-10的整数值，我们也可以把这个问题看成是一个分类问题。不过本章还是把相应变量作为连续值来处理。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "###探索数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "scikit-learn作为机器学习系统，其探索数据的能力是不能与SPSS和R语言相媲美的。不过我们有Pandas库，可以方便的读取数据，完成描述性统计工作。我们通过描述性统计来设计模型。Pandas引入了R语言的dataframe，一种二维表格式异质（heterogeneous）数据结构。Pandas更多功能请见[文档](http://pandas.pydata.org)，这里只用一部分功能，都很容易使用。\n",
    "\n",
    "首先，我们读取`.csv`文件生成dataframe："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "      <th>quality</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7.4</td>\n",
       "      <td>0.70</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.076</td>\n",
       "      <td>11</td>\n",
       "      <td>34</td>\n",
       "      <td>0.9978</td>\n",
       "      <td>3.51</td>\n",
       "      <td>0.56</td>\n",
       "      <td>9.4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7.8</td>\n",
       "      <td>0.88</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2.6</td>\n",
       "      <td>0.098</td>\n",
       "      <td>25</td>\n",
       "      <td>67</td>\n",
       "      <td>0.9968</td>\n",
       "      <td>3.20</td>\n",
       "      <td>0.68</td>\n",
       "      <td>9.8</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7.8</td>\n",
       "      <td>0.76</td>\n",
       "      <td>0.04</td>\n",
       "      <td>2.3</td>\n",
       "      <td>0.092</td>\n",
       "      <td>15</td>\n",
       "      <td>54</td>\n",
       "      <td>0.9970</td>\n",
       "      <td>3.26</td>\n",
       "      <td>0.65</td>\n",
       "      <td>9.8</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>11.2</td>\n",
       "      <td>0.28</td>\n",
       "      <td>0.56</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.075</td>\n",
       "      <td>17</td>\n",
       "      <td>60</td>\n",
       "      <td>0.9980</td>\n",
       "      <td>3.16</td>\n",
       "      <td>0.58</td>\n",
       "      <td>9.8</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7.4</td>\n",
       "      <td>0.70</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.076</td>\n",
       "      <td>11</td>\n",
       "      <td>34</td>\n",
       "      <td>0.9978</td>\n",
       "      <td>3.51</td>\n",
       "      <td>0.56</td>\n",
       "      <td>9.4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n",
       "0            7.4              0.70         0.00             1.9      0.076   \n",
       "1            7.8              0.88         0.00             2.6      0.098   \n",
       "2            7.8              0.76         0.04             2.3      0.092   \n",
       "3           11.2              0.28         0.56             1.9      0.075   \n",
       "4            7.4              0.70         0.00             1.9      0.076   \n",
       "\n",
       "   free sulfur dioxide  total sulfur dioxide  density    pH  sulphates  \\\n",
       "0                   11                    34   0.9978  3.51       0.56   \n",
       "1                   25                    67   0.9968  3.20       0.68   \n",
       "2                   15                    54   0.9970  3.26       0.65   \n",
       "3                   17                    60   0.9980  3.16       0.58   \n",
       "4                   11                    34   0.9978  3.51       0.56   \n",
       "\n",
       "   alcohol  quality  \n",
       "0      9.4        5  \n",
       "1      9.8        5  \n",
       "2      9.8        5  \n",
       "3      9.8        6  \n",
       "4      9.4        5  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv('mlslpic/winequality-red.csv', sep=';')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "      <th>quality</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>8.319637</td>\n",
       "      <td>0.527821</td>\n",
       "      <td>0.270976</td>\n",
       "      <td>2.538806</td>\n",
       "      <td>0.087467</td>\n",
       "      <td>15.874922</td>\n",
       "      <td>46.467792</td>\n",
       "      <td>0.996747</td>\n",
       "      <td>3.311113</td>\n",
       "      <td>0.658149</td>\n",
       "      <td>10.422983</td>\n",
       "      <td>5.636023</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.741096</td>\n",
       "      <td>0.179060</td>\n",
       "      <td>0.194801</td>\n",
       "      <td>1.409928</td>\n",
       "      <td>0.047065</td>\n",
       "      <td>10.460157</td>\n",
       "      <td>32.895324</td>\n",
       "      <td>0.001887</td>\n",
       "      <td>0.154386</td>\n",
       "      <td>0.169507</td>\n",
       "      <td>1.065668</td>\n",
       "      <td>0.807569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>4.600000</td>\n",
       "      <td>0.120000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.900000</td>\n",
       "      <td>0.012000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>0.990070</td>\n",
       "      <td>2.740000</td>\n",
       "      <td>0.330000</td>\n",
       "      <td>8.400000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>7.100000</td>\n",
       "      <td>0.390000</td>\n",
       "      <td>0.090000</td>\n",
       "      <td>1.900000</td>\n",
       "      <td>0.070000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0.995600</td>\n",
       "      <td>3.210000</td>\n",
       "      <td>0.550000</td>\n",
       "      <td>9.500000</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>7.900000</td>\n",
       "      <td>0.520000</td>\n",
       "      <td>0.260000</td>\n",
       "      <td>2.200000</td>\n",
       "      <td>0.079000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>0.996750</td>\n",
       "      <td>3.310000</td>\n",
       "      <td>0.620000</td>\n",
       "      <td>10.200000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>9.200000</td>\n",
       "      <td>0.640000</td>\n",
       "      <td>0.420000</td>\n",
       "      <td>2.600000</td>\n",
       "      <td>0.090000</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>62.000000</td>\n",
       "      <td>0.997835</td>\n",
       "      <td>3.400000</td>\n",
       "      <td>0.730000</td>\n",
       "      <td>11.100000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>15.900000</td>\n",
       "      <td>1.580000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>15.500000</td>\n",
       "      <td>0.611000</td>\n",
       "      <td>72.000000</td>\n",
       "      <td>289.000000</td>\n",
       "      <td>1.003690</td>\n",
       "      <td>4.010000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>14.900000</td>\n",
       "      <td>8.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       fixed acidity  volatile acidity  citric acid  residual sugar  \\\n",
       "count    1599.000000       1599.000000  1599.000000     1599.000000   \n",
       "mean        8.319637          0.527821     0.270976        2.538806   \n",
       "std         1.741096          0.179060     0.194801        1.409928   \n",
       "min         4.600000          0.120000     0.000000        0.900000   \n",
       "25%         7.100000          0.390000     0.090000        1.900000   \n",
       "50%         7.900000          0.520000     0.260000        2.200000   \n",
       "75%         9.200000          0.640000     0.420000        2.600000   \n",
       "max        15.900000          1.580000     1.000000       15.500000   \n",
       "\n",
       "         chlorides  free sulfur dioxide  total sulfur dioxide      density  \\\n",
       "count  1599.000000          1599.000000           1599.000000  1599.000000   \n",
       "mean      0.087467            15.874922             46.467792     0.996747   \n",
       "std       0.047065            10.460157             32.895324     0.001887   \n",
       "min       0.012000             1.000000              6.000000     0.990070   \n",
       "25%       0.070000             7.000000             22.000000     0.995600   \n",
       "50%       0.079000            14.000000             38.000000     0.996750   \n",
       "75%       0.090000            21.000000             62.000000     0.997835   \n",
       "max       0.611000            72.000000            289.000000     1.003690   \n",
       "\n",
       "                pH    sulphates      alcohol      quality  \n",
       "count  1599.000000  1599.000000  1599.000000  1599.000000  \n",
       "mean      3.311113     0.658149    10.422983     5.636023  \n",
       "std       0.154386     0.169507     1.065668     0.807569  \n",
       "min       2.740000     0.330000     8.400000     3.000000  \n",
       "25%       3.210000     0.550000     9.500000     5.000000  \n",
       "50%       3.310000     0.620000    10.200000     6.000000  \n",
       "75%       3.400000     0.730000    11.100000     6.000000  \n",
       "max       4.010000     2.000000    14.900000     8.000000  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "就这么简单，我们通过`Dataframe.describe()`方法获得了一堆描述性统计结果。`pd.read_csv()`读取`.csv`文件。下面我们在用matplotlib看看，获得更直观的认识："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEYCAYAAABSnD3BAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XucHFWZ//HPkysZwoTpRPESWFh+oC4gCOjGFSUgEq/c\nFtcLooCw3mBYxWvcn8b1h4or6OqyoiJJUMALCl4QM+AygigshEuCwAaVoFGEBAgh18lknt8f5wxT\nU1PdXTXTl0nl+369zmv6VJ0656nq6qerq6p7zN0REZHymtDuAEREpLmU6EVESk6JXkSk5JToRURK\nToleRKTklOhFREpOib4kzGyymc1oYv8zzeyfmtT3nmb2wCiX7S/Y/hQz+8Zoxmo0M5toZjub2TPM\nbC8zO9jMjjazE9sd2yAzW2RmJ8XH3zezXdsdkxQ3qd0BSMPsB1xsZp8CPgBMA54LrAUmAw8DdwFf\nBvYGZgG3ALsAE4H5wDeAR4G17v6QmZ0CDL55TAI+ambPBQbitD+4+0/SgZjZs4Cd3f33GfNeGON4\nmbv/pgHrXfSLIJntzeylteKJ22K6u/9nrc7NbCXwp4xZk9z9pbHNacBHgVXA5lg2EJ6rJ4DHzWyi\nu2/LihO4xWt8AcbMTgA+AVSAfuBCdz+/Vtw1eCy4+xsTY3wcuMjdH6vXgZn9LbDB3R8ZZQwyRkr0\nJeHud5nZ2cDz3P0IM9sbOBPoBWa5+zcBzOzzwL1AByG530x4I3gGcCAh0TwIPAScAXwYOAx4CjgO\nMODdwFeBx9NxmNkkYBFwWpVQT45jvg1oRKJvlDeY2TPd/UdV5v8NcNLg0W2GM919KfCwu788PdPM\nkutqwOfc/ZI4rwP4mrufVStAMzsW+Ic6b0gvB84DjnH3+8xsJvA9M3vK3b9eq/9aQ2dMO53wPOex\nCVhsZq/LevOSFnB3lRIU4A3AtPj4fwlH638ClgP3A0uBo4GXAf9JeJEeBXwGuC62vTE+fnXs5wbg\nW8B9wN1x3nXAmvj3XzLieBvw8SoxTgBWAvsDfwUmxul7Ag8k2n08xrwSeA0h0Zwbp/0eWJBouxX4\nEPC7uI7PjtP/Bvg58ABwJyFBApwCfCMjtunArcCEKrF/Ezg0x/Pwm3rTgXfG5+CmWG4mvMHelCg7\nZ2y7W9LTM8a5HnhdatpewEPx8SLgpMS8/vh3Z+DauB2XA/vH6QuBt8bHKwmfEq8C+uJzcWaMfXps\nsw9wd0Zc84G3t/t1sqMWHdGXx87AjWb2KmAF8ClC8r+DcJoGhk7ndAE7xcdbgF8Rjs5/H+t7DHbq\n7ieb2fuAbcDPCEl3obu/qkocbwaqHZkeCfzO3e8xs/uA1wLDTv2Y2VuAVwEHu/tGM+siJOcXEd4g\nJgPXm9lt7n4N4bTTw+7+f8xsMeGTxLnApcDX3f0yMzsE+IGZ7Vtt47n7ejO7HzgU+J+MJs8ALjGz\nLRnzFvvQKZ0ZZnZTRpsNqfod7n5mXOfpwFU1tinAIcAKd0/3k3YA8MvkBHd/0MymmlknI09dJesf\n8/DJ8EzgHODUjLbu7seb2YPAy939L2b2CmAe8APgeODyjLiuAC4kPC/SYkr0JeHu3zGzW9x9rZkZ\n8C7COfedCC/YJ4Cr3X1hPM/73+6+2Mw+AQyeaqjEv2fHv4Mf2R8HXgHMjNP+XCOUv3X3B6vMextw\nZXx8Zaynz/EfC1zg7hvjej1hZq8B/svd+4H+mNCPBK4hJJ8r4rI3AYfGUyF/5+6XxT6WmtlfgOdR\n+5z+XcALyUj07n5M1gJm9o+x38F2f1ej/0FTgKfM7NvAvoQ3V+IbxLMIn1guSy1zIOGTST2TyV5H\nZ+jaysiZ7hviBeHTCW92T+YYa9AVhNN6PyA8f2/N6P9BM9urQJ/SQEr05XK0mf0CeA7Q4+7/A2Bm\nA8Ab4+Mu4KXAIfHo9ADgIsJFO4ATgME7KybEo/njYn1vYvI3szPcPevulazzuZjZNMLR3mvM7JOE\nUxHT49Fs0lTCp4ekSQxPXpaId8CHzvv2E47wq+3X9e7QWU84hVPEDOK1CjO7h/CGCuG6yAvM7DHC\nNRGAre5+ZBzjYcI6nRgf/5u7f8zM3lVlnF0I10nquZtwTWXJ4ISYYDfETy2D2wgzm5xo825Ckv5w\nXLY7x1iDrgW+YGazCUf8DxVYVlpAt1eWy1mE8+d7AMeZ2Q1mdgPwacL5eYB/BS4BvsjQHTWW/hvv\nnFnj7hfG9oOnWubHdj1VYqh21Hgc8Gt3383dn+3uuxHOTb8x1e4XwHvMbIqZTTKzZxCuB7wn3kI6\njfBJ4OfVNoK7rwP+YGZvjStzMOF01QqqvBFFzyTcdfQ0M3u1md1WpfyIcFpsMLk/7O4v93Ax9q9x\n2h2JaYNvSHsSzncPmkHiU0EVjxFOH9Xzb8AX48V4zOyZhOsL58b5K4GD4uMjE8vtB9zg7suBV+cY\nZxPhEx7uvplwfec84DtZjc1sIiPfwKVFdERfEma2P7DS3Z80s9+4+xtS898ZHy4nHEHOcvcLzewI\nwhvEACEJ7g58jXAR9CYAd7/CzF5ASJRbgaPcPesWQoA7zOxQd789Nf1twLdT0y4h3L3x3wwdsX+V\ncEFvBSGZvBv4epx2P+Ei4Nfc/YbYPnmk74n6SQzdbvoo8GZ332ZmTvXTNy9Lx+juP6fGm4qZXU64\naB2rNpWwHWu9oRwU1+VqwimS04Dnm9lHgdtJvdlENwP/UaPPwXhviOv808Q5+U+4+8LY5BvA1WZ2\nDfBrhrbFQsLdOW8nHNHXu211IXCtmX0knmb6LvBj4P1V2h8K3FYvfmmSZl7pJZzrXQ7cA5zd7ivP\nZS6Ec/InxcdrCMknWe4n3HIH4cLZ4J0U3yecMz6NcM77l4Qjxx7glYS7bnqBzxPOX7+T8FH9l8BZ\nGXEcBlzc7u0xiu23N3BNjnZdhIvWywmnZH4EWJy3PLG9743THmXobqXrgRfHbb4f8F7Cm8iXCKdT\njmfoyHiXjLF/DuxZYJ32Jdx19NpYtyZuv5cQThdWm38JMKfdz/OOWgZ30IaLR5hXxB17a9xJ3+0Z\nX6KR8SeeO+8HnuXuKzPmG+FWv/UZ8y4Cvu3uv2p6oA1gZhMIF4c/4u6j+oZugbEqhIvezydcC7nW\nE186itv1RMJdOP2pZZ8PfNoTX1zKMd5ehC/J7QN82N1/PPa1GDGGEV7rP3T372XMPxx4k7u/t9Fj\nSz7NTPQnEu7HPj3W/xXY4u7/3pQBRaQt4h1NPe5+SrtjkWzNPEd/D3BuPILZDLyO7PuTRWQ75u7P\naXcMUlvTEr27329m5xHO9W4g3ANc9T5eERFpjqaduhkxkNlngD+6+0WJafrP5CIio+Dute7sGqap\nt1fGH4l61Mz2INxR8PfpNkWCbRczW+DuC9odRz2Ks7EUZ2NtD3FuDzFC8YPkZt9Hf2X89bytwHs9\nfJFFRERaqKmJ3t1f0cz+RUSkPv0EQj697Q4gp952B5BTb7sDyKm33QHk1NvuAHLqbXcAOfS2O4Bm\naNnF2MzBzXx7OEcvIjKeFM2dOqIXESk5JXoRkZJTohcRKTklehGRklOiFxEpOSV6EZGSU6IXESk5\nJXoRkZJTohcRKTklehGRklOiFxEpOSV6EZGSU6IXESk5JXoRkZJTohcRKTklehGRklOiFxEpOSV6\nEZGSU6IXESk5JXoRkZJTohcRKbmmJnoze7+Z3WNmy83scjOb2szxRERkpKYlejN7LnAWcIi7HwBM\nBN7crPHKzMzmmc3sCcXmFZ0/XpjZfLOZa0Kx+TmXGbZuRdc1a8x2bK9665FnPRu9H7RqO4zmeZcG\nc/emFOC5wB+BLmAS8BPgqFQbb9b4ZSnAPOjcCIs8lM6NwLy888dLAeZDpyfidGB+sXXv2Aydm/Ou\na5UxF7Z6e2U8R5vDulRdr9T8zo1xXRq2H7RqvxnN866Sa7t6ofZNDuZs4CngUeBbYw12RyxQ6Qkv\nEI9lkUOlJ+/88VKgsiYjzjXF1n2OF1nX7DG7+lq9vbKfozl11mtOOsas7Tfq/aBV+81onneVPNsV\nL9J+UqM+GaSZWRdwDLAn8CTwfTM7yd0vS7VbkKj2untvs2ISEdkemdlcYO6oO2jiO84bgYsT9ZOB\nC8fyrrQjFnTqRqdudOpGZeR29ULtmxjIS4B7gGmAAYuB940l2B21hBdlpSeUkS/GevPHSwkv+sqa\nUPK92NPrVnRds8Zsx/aqtx551rPR+0GrtsNonneVutvUi7S3uFBTxNMybwL6gTuA0919a2K+u7s1\nLQARkRIqmjubmujrDq5ELyJSWNHcqW/GioiUnBK9iEjJKdGLiJScEr2ISMkp0YuIlJwSvYhIySnR\ni4iUnBK9iEjJKdGLiJScEr2ISMkp0YuIlJwSvYhIySnRi4iUnBK9iEjJKdGLiJScEr2ISMkp0YuI\nlJwSvYhIySnRi4iUnBK9iEjJKdGLiJScEr2ISMkp0YuIlFxTE72ZPc/M7kyUJ82su5ljiohIiru3\npBDeVB4Gdk9M81aN38T1mg+VNaEwvxHt022AeVDpCYV5wBLoGgiFJcBq6PJQWJ01RsYy6T7T9fWh\nv+kOu/TDLhvC4y4HVqTbxzE2JeLYBNyeqN+eMT8d0/rE/PWxz1WJaasyts2KxPwVGfVhY8Y+h42T\nEefaRH1txvZNxz0sxjhGOo50n+n1SNez+lwIXX2hsHAU+17N5zy7bcefYdetg2OO8jVSdRyVUecd\nL9S+hYEdDfxqLMGOtxJeaJ0Oi2Lp9FovuDzts9t09A3VJ/cPnz/ZR7afnB5jxcg2E/uH6h2boXPz\nUH3iQGhzjsOsxHKz4rROh4nbEv1tBLYMH2NiRlwdqbrVWQ+2ZcRdY5mpOcZM99lRZ5ms+cm4s9aT\nvvrPydSica/P8TzX2vfmheep2nPeuTHxhh3bpp//Ti+a7EeOOzSOyphyjxdq38LALgHeO5Zgx1sJ\nR1KLHDyWRQ6VNWNpn91mTqI++MKrVh9MyMl6V0abSqI+JzW/EusnZCx3QsbyWWNUMpad49VjyFqP\nenGnl5mdY8x0n+l1Ty+TNT8ZQ9Z6ZsWdfk5mjzHurD5r7ns99de70jO8bdbz39VX8DXSU20clbHk\nHrxI+0m0gJlNAd4AfCRj3oJEtdfde1sRk4jI9sLM5gJzR7t8SxI98BpgqbuvTs9w9wUtiqEJHr8A\nus8dqncD6y4YW/usNv1bYfHkUH9yG3RPHJr/ZGyTbL8JWJwc4wHo3md4mw3bYHHsZ9mW2MfU2KdD\nt8EZwAcTfX8QeMfg8gOwOF7M794E6yZC95TacfWn4noqUc9qv24AuicMn7ahxjKrc4yZ7nNZnWWy\n5teNeyt0Tx4+Lf2cbEnUc8W9Abp3rt1nzX3vfOg+DJgW1yv1nHdvgnXnD297xrThz383sO6y6mPk\nGTc5juQVD4B7B+tm9smiHbTiY8Z3gHeM9ePHeCzoYqwuxupibKFx2/2aLUMpmjstLtQ0ZrYz8BCw\nl7s/lZrn7m5NDUBEpGSK5s6mJ/qagyvRi4gUVjR36puxIiIlp0QvIlJySvQiIiWnRC8iUnJK9CIi\nJadELyJSckr0IiIlp0QvIlJySvQiIiWnRC8iUnJK9CIiJadELyJSckr0IiIlp0QvIlJySvQiIiWn\nRC8iUnJK9CIiJadELyJSckr0IiIlp0QvIlJySvQiIiWnRC8iUnJK9CIiJdfURG9mu5rZlWZ2n5nd\na2ZzmjmeiIhkcPemFWAxcFp8PAmYkZrvY+h7HlR6QmFezmXmQ2VNKMzP0X4bdHkobIvTViWmrQpt\npnsouzrsMhAedznQl+5jqL6rwxSHqZ6Y3x/H2JqYtrV6H+n6DIed031uy1qXYvUpDlMGasdU6YFp\ncb3yjjEp2Wd/KCO2RdG4Vyfqq4G1ifra2Gd/atz09l6fqK8HlkDXQCgsydgHbk/Ub6+yn2xK1Del\n90VgRWL+ivR6xD4XQldfKCws+hrJGHPEayhjmcKvs4w4hm2/ZuacHaUUzZ3NDGQG8IdGBptYbh50\nboRFHkrnxno7YdixOz2xjNdK9iFpjGifMc3i31mJ6RWHc2LCTbaf6COX78gxxtREfXKdPjoy5mf1\nWWuZrD6mpuqT4+NzYv2c1DZI95EV9+Q6Y6bjzmqTXI/09k6PkdVn1jIT6sxPP2cjxthSfHsm61n7\nCZsypi3M/xrp6MtYj77Ua2h+6nW1GTo2F3mdZcSxJCNuJfsxlvGU6A8Cbg1HIdwBfAPoGEuwQ8tV\nesJO47Escqj01FlmTcYya6q37/KR7bOmVRzmZEw/wWF2anolo92cHGPMTtRn1ekjPWa1PpPLpOPP\nWp/Zqfqs+PiExPrW6iMr7ll1xkzHndWm3rrPqtNn1jKVOvPTz1m9MfJsz2Q9az/J3B/78r9G6m27\n8HrI0abm6yzjdTSQEfdAuxPl9l6K5s5J1GFmxwA/dfeBem1TJgEHA2e6+21m9iXgo8AnUv0vSFR7\n3b234DgiIqVmZnOBuaNdvm6iB94EfMnMrgQucff7c/a9Cljl7rfF+pWERD+Muy/I2V/C4+dD92HA\ntFDv3gTrzq+zzAXQfe5QvRtYd0H19k8MQHfiYnU3sC5j2lPAvcAHE8t+ADgVuCa2GfRkqt5NOEW8\nuM4YWxJt6vXxeMb8rD6TyyxLLZOup2PoJpxmXgzsFetnMHwbpPvIinuwj2pjpuPOapNcj9V1xsjq\nM2uZ9XX6TD9nI8bog+4pteNOb896z/G6zdC9U2raZVSVfo0s2wrdk1PrsRUWx2ndm8Lroftfh5bp\n3hLXdWqiTZ3XWdoT10H30am4ryvWh8QD4N7Bupl9smgHeT4mzADeDdwC/Ab4Z2CXHMvdCOwbHy8A\nzhvLx4/UsroYq4uxuhhb4DWSMaYuxm6npWjutLhQXWY2CzgZ+BfCIew+wJfd/cs1ljkQuBiYAvwe\nONXdn0zMd3e3XAGIiAhQPHfWTfRmdixwCiGxXwoscvdHzawDuNfd92xVsCIiUjx35jlHfwLwRXe/\nMTnR3Tea2elFAxQRkdbK883YR9JJ3szOA3D365sSlYiINEyeRP+qjGmvbXQgIiLSHFVP3ZjZe4D3\nAnub2fLErF2Am5sdmIiINEbVi7FmNgPoAj4HfAQYPPH/lLs/1pDBdTFWRKSwht11Y2ad7r7OzGYC\nIxq5++OjD/PpMZToRUQKamSiv8bdX2dmK8lO9HuNOsqhMZToRUQKavh99M2kRC8iUlzD7qM3s4Nr\nLejudxQJTERE2qPWqZteMk7ZDHL3I8Y8uI7oRUQK06kbEZGSa8ZPIGBmBwAvAJ7+mVR3v7R4eCIi\n0mp5/vHIAuBwYD/CD6y/BvgV4QfORERknMvzEwgnAkcBD7v7qcCBwK5NjUpERBomT6Lf5O7bgP74\nbdlHgd2bG5aIiDRKnnP0t5lZF+Gfe98ObAB+3dSoRESkYQrddWNmexH+heCyhgyuu25ERAprxn+Y\nOpzsn0C4MaN5IUr0IiLFNSPR/5ShRL8T8BJgqbsfOeooh/pWohcRKajh99G7++tTA+wO/McoYhMR\nkTbIc9dN2irCl6dERGQ7kOcLU19JVCcABwFLmxaRiIg0VJ7bK+9PtHsMuNzd9a8ERUS2E7V+pngK\n8Hng7cDKOPlZwFeAm83sIHe/q94A8R+XrAO2AVvd/SVjjFlERIpw98xCSOgXE+6bH5zWSfji1EXA\nymrLpvp5EKhUmed5+sg5zjyo9ITCvJzLzIfKmlCYnzF/E3R5KGyK07Ympm0FtiXq24rVJ8UyND+O\nMYY+R9ar9NmXqPcBWxL1LRntVyfqq4FVifqqOMbaxLS1GWOk+xzWR3qM2OftiWm3Z4yxELr6QmFh\nej/I2i/Sz/tQm12Xws6x7Lq0Wh+j2deK7q85414CXQOhsKRRr6UdoTTiOWxz/F6ofY2Ofg9MyJg+\nEXgCeGnOgB4EZjYi2NpPWudGWOShdG6s9+SFF0qnJ5bxZLIPSX7E/G3Dp+Ej20xM1Cek5k/NaD+5\nzhgdGct0pJepMWZWn+k403HlGbNo3JNzjDE13WdfjjbpPvsT9c3QuTm1XyzMiKNvqF5Jx5nuY3OY\nln9fK7q/VpmfFXd6WyjZNylfjLfSyES/YjTzMtr+AbgzHo2dMZZgq49R6QlPmMeyyKHSU2eZNRnL\nrBma3+Uj56enVTLaVGrMn53RfladMeZkLDOnwJhZfaaXSceVZ8yicc/KMcbsHHGn21Rq9Jk1Rldf\nsXXNtf1r7mtF99fs+Xni7hpodwLaHspo8sV4K0VzZ62LsfeZ2TvcfXFyopmdDNxXY7m0l7n7w2b2\nDOA6M7vf3W9K9Lcg0bbX3XsL9C0iUnpmNheYO9rlayX69wE/NLPTGLqd8hCgAzg+7wDu/nD8u9rM\nriJ8s/amxPwFBWPO8Pj50H0YMC3UuzfBuvPrLHMBdJ87VO8G1l0wVH9iM3TvlJo/AN2J7x48Hqcn\n22wABt8b16bmr85ovynRPmuMZRnL9KeXqTFmVp/pONNx5RmzaNxP5hhjS7rPrdA9uU6bRH0Z0L2N\ncHoRWLYljjE1tt8E674L3aek4tgKxHE+kFrXEX1sifOTfdbZ19Lq7a+Z87PiTvTZDay7rlgcO6rR\n5Iv2igfAvYN1M/tk0Q5qfTww4JWEvegs4JUFP150EC/mAjsDNwNHj/bjR52xdDFWF2MXpveDrP0i\n/byji7E7XGnEc9jm+L1I+6b+z9j4a5dXxeok4DJ3/2xivrt+60ZEpBD9c3ARkZIrmjtH81s3IiKy\nHVGiFxEpOSV6EZGSU6IXESk5JXoRkZJTohcRKTklehGRklOiFxEpOSV6EZGSU6IXESk5JXoRkZJT\nohcRKTklehGRklOiFxEpOSV6EZGSU6IXESk5JXoRkZJTohcRKTklehGRklOiFxEpOSV6EZGSU6IX\nESk5JXoRkZJreqI3s4lmdqeZ/aTZY4mIyEitOKI/G7gX8BaM1VBmNt9s5hqzXZ402/UBs5k9ZrYw\nTJu5Jsy3hWaVvlBs4dAyT88fVs8YY4VZxUOxFcPHrTrGvBDLzJ7w2JaYVQZCsSXp+VXWLd1HzTFG\nuf3SfdaMM92+SpyFtm+edS86f7zaXuOWFnD3phVgNnA9cATwk4z53szxxxj7fOh0OMdhlsOiWAan\nLXLoiPVFVeqdcVqyzvzEGCtGtmd1jj63DtWn9me035ZovxGYl1q3eWH60236R8Y5eWutPnJsv4X1\nt8XU/toxsGR4nB19w/ucmrFthrZvlbjS6z5s3erNH69le41bZdTPtxdq3+Rgvg+8CDh8+0v0lTXh\nBXNCfOF4LIPT3GFOal66vihOS9Yra4bG6Mpon55Wr8/ZOdpXelLr1lM/zlk1+6i//br66o8xu878\nroHa65Ze9+Hbt8rzmrHuQ+tWb/54Ldtr3Cqjfb7xIu0nNf4zQmBmrwcedfc7zWxujXYLEtVed+9t\nVkwiItujmEPnjrqDJr7jfAb4E/Ag8DCwAbh0LO9KLX7H1KkbnboZ1bq3aX/dLuNWGfXz7YXatyio\nw9nOTt3E+OaHUzjTn4QZD4SPxywM0yprwnwWhtMUXX3h8eAyT88fVs8YY0U4XdPlwIrh41YdY16I\npdITHrMknOboGgiPh8+vsm7pPmqOMcrtl+6zZpzp9lXiLLR986x70fnjtWyvcauM6rn2Iu0tLtRU\nZnY4cI67H5Oa7u5uTQ9ARKREiubOliT6qoMr0YuIFFY0d+qbsSIiJadELyJSckr0IiIlp0QvIlJy\nSvQiIiWnRC8iUnJK9CIiJadELyJSckr0IiIlp0QvIlJySvQiIiWnRC8iUnJK9CIiJadELyJSckr0\nIiIlp0QvIlJySvQiIiWnRC8iUnJK9CIiJadELyJSckr0IiIlp0QvIlJySvQiIiXX1ERvZjuZ2a1m\ndpeZ3WNmC5o5noiIjNTURO/um4Ej3P0g4CDg1Wb2943o28zmmc3sCcXmNaLPdoyRZ9zq9elLzbqW\n5omv0euSp7966zHWGEQkJ3dvSQE6gKXAixPTfJR9zYPOjbDIQ+ncCMxrcLxNHyPnuJuhY/PI+jkO\nszxPfI1elzz95ViPlmxPFZUylqK5sxUBTQDuAp4CPjuWYIeWq/SEZOGxLHKo9DQ27uaPkX/cORn1\nEzxvfI1elzz95VuP5m9PFZUylqK5c1LTPipE7j4AHGRmM4CrzGw/d//t4PzUefted+9tdkwiItsT\nM5sLzB11By1+F/q/wDmjfVdKLKdTNzp1o6Kyw5aiudPiQk1hZrOAfndfa2bTgCXA59z9Z3G+u7uN\nsu95UDkn1B4/392XNCjslo6RZ9zwN6u+ZSZMBiY8Vi++Rq9Lnv7qrUertqdI2RTNnc1O9AcAi4GJ\nhHP133X3/5eYP+pELyKyoxpXib7u4Er0IiKFFc2d+masiEjJKdGLiJScEr2ISMkp0YuIlJwSvYhI\nySnRi4iUnBK9iEjJKdGLiJScEr2ISMkp0YuIlJwSvYhIySnRi4iUnBK9iEjJKdGLiJScEr2ISMkp\n0YuIlJwSvYhIySnRi4iUnBK9iEjJKdGLiJScEr2ISMkp0YuIlFxTE72Z7W5mN5jZb83sHjPrbuZ4\nIiIyUrOP6LcC73f3/YA5wPvM7AVNHrPhzGxuu2PIQ3E2luJsrO0hzu0hxtFoaqJ397+6+13x8Xrg\nPuA5zRyzSea2O4Cc5rY7gJzmtjuAnOa2O4Cc5rY7gJzmtjuAHOa2O4BmaNk5ejPbE3gRcGurxhQR\nkRYlejObDlwJnB2P7EVEpEXM3Zs7gNlk4KfAte7+pdS85g4uIlJS7m552zY10ZuZAYuBx9z9/U0b\nSEREqmp2oj8MuBFYBgwO9DF3/3nTBhURkWGafupGRETaq23fjDWz98cvUS03s8vNbGq7YqnFzM6O\nMd5jZme3O55BZnaJmT1iZssT0ypmdp2ZrTCzHjPbtZ0xxpiy4nxj/BLdNjM7uJ3xDaoS57+b2X1m\ndreZ/dDMZozDGD8d47vTzJaY2bPbGWOMaUSciXnnmNmAmVXaEVsqlqztucDMVsXteaeZvbqdMcaY\nMrenmZ0D9SIXAAAFT0lEQVQV9897zOy8Wn20JdGb2XOBs4BD3P0AYCLw5nbEUouZ7Q+cDrwYOBB4\nvZnt3d6onrYQSO+EHwWuc/d9gV/EertlxbkcOJ5wWm+8yIqzB9jP3Q8EVgAfa3lUw2XF+Hl3P9Dd\nX0S46eETrQ9rhKw4MbPdgVcBD7U8omxZcTpwgbu/KJbxcJp5RJxmdgRwDPBCd98f+EKtDtr5WzeT\ngA4zmwR0AH9uYyzVPB+41d03u/s24JfACW2OCQB3vwl4IjX5GMLFb+Lf41oaVIasON39fndf0aaQ\nMlWJ8zp3H4jVW4HZLQ9seDxZMT6VqE4HBmizKvsmwAXAh1scTlU14sx9N0srVInzPcBn3X1rbLO6\nVh9tSfTu/mfgfOCPwF+Ate5+fTtiqeMe4OXxlEgH8Dra/GKvYzd3fyQ+fgTYrZ3BlMxpwM/aHUQW\nMzvXzP4IvJXxcUQ/gpkdC6xy92XtjiWHs+LpsG+Oh9OfVewDvMLMbjGzXjM7tFbjdp266SIcfe5J\n+EmE6WZ2UjtiqcXd7wfOI3yEvxa4k3FwxJSHh6vsutLeAGb2caDP3S9vdyxZ3P3j7r4HcBnhlOi4\nEg+S5gOfTE5uUzj1fBXYCzgIeJhwQDoeTQK63H0O8CHge7Uat+vUzVHAg+7+mLv3Az8E/qFNsdTk\n7pe4+6HufjiwFvjfdsdUwyNm9iyAeFHu0TbHs90zs1OA1wLj7kAkw+XAP7Y7iAx7Ew7q7jazBwmf\nipea2TPbGlUGd3/UI+Bi4CXtjqmKVYS8ibvfBgyY2cxqjduV6B8C5pjZtPilqqOAe9sUS02DO6OZ\n7UG4gDguj+qiHwPviI/fAVzdxljyGq9HdsQ7Lj4EHOvum9sdTxYz2ydRPZbww4Hjirsvd/fd3H0v\nd9+LkKQOdvdxdyCSumvpeMKNA+PR1cCRAGa2LzDF3R+r2trd21KABYSdcjnhwuHkdsVSJ84bgd8C\ndwFHtDueRFxXEK5v9AF/Ak4FKsD1hDtEeoBdx2GcpxEuEv8J2AT8lfDzGOMxzgcIByV3xvJf4zDG\nK+Nr6G7gR8Czx9G23DK4b6bm/wGojKM4k9vzUsIXPO+OyXS3cRTn09sTmAx8Kz73S4G5tfrQF6ZE\nREpO/0pQRKTklOhFREpOiV5EpOSU6EVESk6JXkSk5JToRURKToleSsvMjos/ifu8WN8z66dzc/a1\nsshP65rZKWb2ldGMJdJoSvRSZm8Bbop/x8op9i1efUFFxg0leiklM5sOvIzw/wRG/K8DM5toZl+I\n/1TmbjM7M05/pZndYWbL4q8XTkksdpaZLY3zBj8lVMzs6tjHb8zsgFasn0gRSvRSVscSflrhAeCx\njP9k9c/AHsCBHv6xyGVmthPhnzz8k7u/kPALge9JLLPa3Q8h/MLhB+O0TwFLYx/zCV+hh3H8Gz6y\n41Gil7J6C/Dd+Pi7sZ48nfJK4Gse/7GIuz8BPI/wq6q/i20WA69ILPPD+PcOwq8xQvjU8K3Yxw3A\nTDPbpaFrIjJGk9odgEijxYumRwD7m5kT/lXlAHBhummqnj6vbqlpW+LfbQx/7dTrR6StdEQvZXQi\ncKm77+nhp3H3AFYSTtUMug54l5lNhKf/Gc4KYM/E/wU+mfDvI2u5ifhb9WY2l3B6Z32jVkSkEZTo\npYzeDFyVmvYDwj9LHzzavpjwryyXmdldwFs8/Ob8qcD3zWwZ0A9cFNsnj9KT/71rAXCImd0NfIah\n/weg//Al44Z+plhEpOR0RC8iUnJK9CIiJadELyJSckr0IiIlp0QvIlJySvQiIiWnRC8iUnJK9CIi\nJff/AULvdxmHqDsYAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x830ea20>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(df['alcohol'], df['quality'])\n",
    "plt.xlabel('Alcohol')\n",
    "plt.ylabel('Quality')\n",
    "plt.title('酒精度（Alcohol）与品质（ Quality）',fontproperties=font)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "散点图显示酒精度（Alcohol）与品质有比较弱的正相关特性，整体呈左下-右上趋势，也就是说酒精度较高的酒具有较高的品质。挥发性酸度（volatile acidity）与品质呈现负相关特性："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEYCAYAAABfgk2GAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcHHWd//HXZyaTYxImTCcCQrjMIiCyoBzGBSVRMYgr\nukRXWUSBBVkRZSG7K87PI+wq4gHeuyqsRBQRV/SxoiBhlZFjOQMRFFDkFhE2BwlkMmSS+fz++H47\nU6mp7unumeqeSb2fj0c9eqrqe3yquvpT1d/q6TZ3R0REtn1trQ5ARESaQwlfRKQglPBFRApCCV9E\npCCU8EVECkIJX0SkIJTwxxkzazez3ePflrF+2LI62j7MzF5dZf0sM/vbRttvIJ6lZnZ8HeVPLcdn\nZr1mdlgOMf27me2VsXy+mV2XjmMiiMfUDDPbwczmmtnBZvYmMzu61bGVxefzr8xsqpn9d6vj2VZN\nanUA2zoz+yRwFfBq4O+BZxOrtwN+7O7nmtmlwCnAy4GzgBOAs8xsR+Bj7r4x1vm4mfW6+6/M7FXA\nb4Fr3P01iT4/D7wemAmsB+4G/gH4f8ApZvYBd/9aLHtiLAfheDjHzHYBBuOyh939qozt2gmY7u4P\njWL3eJwqMrMvuPtZAO5+UT11GwrI/fRKqxJltsRhZq8BSu4+YpIysynAfu5+V5UyS4A73f2nVcpM\nBZ4G7kmtmgI85O7HxXL/CvwN8AzQH6f1wJo4PVmlj8Pc/eYRtuc0wrE6DdgAfNLdv1utThUO4O79\nwFsTfWx5/kdiZvsBT7j7ugZj2ObpCj9HZvbPwNHAR4BO4FRCgn5vTNDHA38yswOAyTGpHwCsAHD3\nC4FHgENSTb8qPn6ZocS8hbv/E/BR4DLgfOC9wLeA7wN7ACeY2Zti8VOBO4GphBft24A7Yp93An/I\n2K5JwNJYfrRGesdyxhj0MRYqxfl6wkl6RO7+AnCmmR1cpdjuwKfN7JYK086x3F3u/prkBLwjI+Yz\n3f317v5mwnH4pLuf6e5LUifQoUpmZwL7V9sWM3s34QLlde6+O7AQ+Gcze+NI+6FO9Tz/bUDmNkmg\nK/x8fQPYEbiWkBQ+DxwI7JcYmfkZsAuwu5ndArwUeNTM3p5syMzuAn5PSNj7mtlK4MWx7f3N7MZY\n9FuEk8A/Ed5B9MUYHo8xdMUyLweuATYSrv4Pjn8/E9t5Raz3M+D+1Ha9C7jR3f+UivEm4OPu/ksz\n6wAeBfaObV9AeCfxIHCyuz8Vq3ms+4/A6YSr1C+5+4VmdjvQbma/B94PvAYYcPdPpfo9EvgsMB34\nvrt/PLV+F+B7cT+vAv7G3f8Uk+fXgZfFdvc1s17g/7n7zXHI4wJgALg50d6SuOxR4IOAm9mhwDrg\nhnIiNbObgTPc/e5EOB8DLgYqJcbdgAXuvrLC+vIVfuaqjGVHm9m58e9OYGczOzzOr4sngmTb2wHH\nufu8Sv1HnwDeVj4G3P2xeKI4F1iW2o97ANe5+16VnotUDAPu3pF6/j8DnOvuc2KZI4EPuvsx5Xru\nfq+ZrTWz17r7DSPEX0hK+Dly93VmtoHwwvqCmb2Y8EJ5vbv3pIpfDWBmK4BD3H3YlbuZfRd4GPgJ\nIQH+rbvfamY3poZ0PgxcHmfXEhLadOBXwHWEq6DPJuI8wcw+AGyOcRhwibsfWWHT3kVIdGlXEN6O\n/5Jw5Xsj0AFcChzp7r8zsw8R3pmkr0bvcPeXmtls4CEz+7K7Hxpf/C+N23U4qWEcMysB/wYcQTi5\nXW9mB7r7ikSxQeAkd3/YzD5HeFdzLnAJ8BN3P8bMZpV3ByGBl4BvAq+N9S5O9O1ht/llZvYXhJPF\nefEE8X7gojjk1Z1K9rj742Y22cx2dPenM/bhi4BrKtyqOd/dr4zbs0/iJF82mfAOMuln8R0fZvZy\nwhX/qVmNR0cRLiIqMrPphG1L93U7Q+92Kg25VXouhsl4/k8zs4PcfTlhqOqyjGqXE45PJfwMSvg5\nildLc4Fvmtn5wKHu/i9mdk6F8gsIV8QrzGxVXDwbuMXd30dIoDsBq4Hz3P3WKt3vB3wXODm28VXg\nHMI47g+AOeVu4+Nq4LXArLis4vgu8BJ3fyRj+Q8JV8JnAosIL8i/IiTz38Uy3wSSV+Dl/lfFK9GX\nEa7yXwQ8RXUW298buC0umw68hDgsFj0NvDueCF8N3GJm04CD3X0hgLuvSpS3WO42d384LrucsP/S\ncSf/vg74Rmz7rYQhtCy/JgyZDEv47v6XmRtqtpiwX4hDfy+u0HbSZGAgvvPaknzjiWJ3wpV8epz+\nAMJQXjWTyE7mTsYQY8ozwPHJ52KE8kmXA8cAy4E3AIszyvyacAEgGTSGn6+vERLvl4A/Et6e3ggc\nbGY3xin5SZOzCYniVndf4O4LCFds3wRw928kynaU2yAO6cSp/Hb9SXe/hnAlvwI4FLiPMOxwBvBf\nsVxbvLo/GfgLwpXyEcBOZlbpSjDz8jMO0zxuZgcBhxOGjNoZngQ2J6vFq/pfEk4WpwMrK/WRoR34\nhbvvG6fd3P1HqTLnAQuACwlDOG2EZFgtOU0hDNuUVRpGGdoQ9wHCENgbCfdCLq9Q9Hlgxkjtpcwk\nnJQxs1WJ5/t6M9vFzJ6O87eZ2SWxzozYl8d3gMcBV8a/s66OIQwDPl8tEHdfCzxvZvukVs1j6ES7\nifDcQHiXV/Yphj8XtboCeLOZzSPc2M66h9TIvi0MJfwcuft7CMMvv3X3GxM31+5M3Gy7GcDMjgKe\ncfdlwDNm9r54VfeYuw+74nL36xLt3Zto76ZY5Agzu57wwtqbMD6/mfAivMrd++Oww0oPn9j5FmGs\n/migh5Bwl1XYtGqJ8grCW/Tr3X0TcCtwmJm9NK4/Ffh5orwRrjZXE66Q5xLexZQNmFmXDY1xJE8E\nTrjBfISZzYXw8cmMmPYjnHz+QBiy8Ji0HrP4sdDEzdByu3cCC2IynUT4hFUy5rINQCkxfznwbmCm\nuz+YEQvADgzdKyH2f5KZ3VFh+jrhXdrqWPyuxHNfdnWcPzaxbA/CfYaynQj3CJLbmbaK8O5qJOcC\nF5vZDjH+lxCOtc/E9Y8S7gMBvC5Rb9hzMUI/A2bWBRDH+tcRhhO/V6H8DsD/1RB/IWlIJ38GYGY/\nYOiFdEBMxgAPAJ8ElhCuCiEMv9xOeDFU/Nz8CN4Tx8y3Bz7h7h8zs5MICan8OfM3EYaJcPfLzWxf\nwo3hAeAN7v5EhbbvMrODs05EwJWEdzQLYrvPmNnJwJUWPpb4a+C0RHknXBU+REgSPweS/V4M3Auc\nmCg/VDncfD0LuM7MBoG7gN5UTF8jvEv6CGEIoXzc/x0haf0r8Gdgy7utONa+JJZfE7er/O4pOT59\nFWHMfTd3fwdhf34X+GLGvik7iNSnT9z9EsI9hWHiye4mhpJ3W9yXWe+CysdbG7Cjuz9rZt+O695N\nOPm+P25X1vN7M2F8vNK7k3K8S+OQ5a0WbiI/Dyx29/+JRT4PfN/MXke4kCjvr0rPxVbNJ/6+GLjX\nzN7r7r2EC4pPET55luUwwr6SDOY5fh++hbv2pxAOwovc/Uu5dTYOmdmXgCMJN/5WJpYvc/c3JuYv\nJby9fQkh6XcBnya8Aysno2+7+0/M7BOEhPbJRFdzCQkTwqdgfke40usjvKAeIwwprSUkrn8gJMb3\nEK7ITgR2JZxkvkv4SObbCZ/q+KG7fyW1XYcDJ7r7KQ3vnG2Ymd0LHOXuw+6DmNkRhLHzfxihjf2B\nHxHeQUwlDFu9P677P+A3sagT/mfjdsKQ3RTC1fM1hGG8HxBOVG+Lyz4Xy59GOFl9Nr4TK/fbBvwv\n4Sb7czVu72GEk9UJ7n6bmZnnlFgs/MPbG+I9raz11wKnVLlYKTZ3z2Ui3K2/l3CwthPfrufV33ic\ngLZ6yhE+4bFPxvrDgX1HEYcBUzKWz4jPzx5V6s2osO7rwOGt3sfjbSIMiV1dYd00widgZjUhjj0J\n7yiPJ5y8Z6TWdxA+5ZVV9wjga3X2dyDhPswDwEE5bdMkwtX7oRXWnwB8pNXHwHiecrvCt/A58qM8\nXgWa2UeBF9z9c7l0KNJiZnYBYVz6GB/dfyBLioWPNK8gjBR8tNXxTFR5Jvx9gP8mjEH3A78Abnf3\nM3PpUEREqsrtpq27P2BmnyF80qP8fS4jfUZXRERykutN2606MjsPeNzdv55Ypl9QFxFpgLvX/825\ned4gAHaIj7sRPprVlVrvrb6JUSHuJa2OQTEppiLGpZhqjskbqZf35/B/aOE7SgaA011fWyoi0jK5\nJnx3f22e7YuISO301QrZelsdQIbeVgeQobfVAWTobXUAGXpbHUAFva0OIENvqwPI0NvqAMZK027a\nZnYe/iGv4Z/sExEpokZzp67wRUQKQglfRKQglPBFRApCCV9EpCCU8EVECkIJX0SkIJTwRUQKQglf\nRKQglPBFRApCCV9EpCCU8EVECkIJX0SkIJTwRUQKQglfRKQglPBFRApCCV9EpCCU8EVECkIJX0Sk\nIJTwRUQKQglfRKQglPBFRAoi14RvZmeZ2W/M7F4z+56ZTcmzPxERqSy3hG9muwAfBA5y9/2BduBd\nefXXbGa20GzWsjDZwux13cvNZiwfeiz/HeqY2SVm2w2YlQbNZj4Zl23VbsZ8j9mslWGyntHGOlKZ\n0dQdjyZSrCJjzt1zmYBdgMeBbmAScBXwhlQZz6v/PCdgIXT1wVIPU1cfsDB73WyHxQ5dDiVP1NkE\nU+L6Lcs2Q2d/Yr5/6/nOjaGdLesd6Gk01pHKjKZuq5+jRveFJk0TYWo0d+Yd1JnAc8AzwHfGKuhW\nT1BaFhKGx2mpQ2lZ5XXHOszz4cvnZCybV2U+q43SykZjHanMaOq2+jlqdF9o0jQRpkZz56Q83jUA\nmFk3cAywB7AW+C8zO97dL0uVW5KY7XX33rxiEhGZiMxsPjB/1A3leAZ6B3BxYv4E4GtjcZZq9YSG\ndOpqf7xMEylWTZqqTY3mTouVx5yZHQp8CzgE6AeWAre7+9cSZdzdLZcAchZu+JUWh7nVF7j7tcPX\nDc6CAaCD8Ej8u20VrL4AeBfMeDd0tMPmp2DdyaHMULsZ8wdB6ew4f6G7nzeaWEcqM5q649FEilWk\nkkZzZ24JH7YM17wT2ATcBZzi7gOJ9RM24YuItMq4TPgjdq6ELyJSt0Zzp/7TVkSkIJTwRUQKQglf\nRKQglPBFRApCCV9EpCCU8EVECkIJX0SkIJTwRUQKQglfRKQglPBFRApCCV9EpCCU8EVECkIJX0Sk\nIJTwRUQKQglfRKQglPBFRApCCV9EpCCU8EVECkIJX0SkIJTwRUQKQglfRKQglPBFRApCCV9EpCBy\nTfhmtreZ3Z2Y1prZh/LsU0REspm7N6cjszbgSeBQd38iLnN3tzHu5xLoPj7MrbnM3U+qUnYhlBaH\nudUXhMfSYti0H2zcEaa1weanYN3JwKeg+yDoAzpjC88PQofBZIN1cXkHMBDXOzApzk8GjFDfgKnx\ncY1Dt8HzwIxYb11sZyqwCWgHBmN75TbbYlsdcVqz3N0PjtvVA6WzYbAb+ttgGrBmE3R7iKW9LTSw\n2UJbGzfB5A2w8WmwdaG9vp1g+o4hnrUGM9tC3xuBNoe21TDpLlj9JJTeEvfhheGx+5ywMzbEbe9q\nj8/Hcnc/eGi/D84K66esgtUXuPu1lZ6b5LqRZD+vU74KnbsD/bDmfHc/r5G26olDJC8N5053b8oE\nvBG4KbXMx7iPS6DLYWmcuhy4pELZhdDVN1S2sx+6+reuu7i8bjDML/Lh7U9x2M2h02F2Yt0Mh1Js\nY3ZGneT8yzLa7cioW8oo1xnLdTlwJ9AzvEw57hel2pud6GNxov2s7VxUJYbyfpqSWteRUZbfb73f\ny3139QELs5+boXU1HAPpuv3QMZARR08DbdUchyZNeU6N5s5mBvgt4PSxCLpyH90bwwvT47TUoXtj\ndtnSsq3LzvPhdY9NrZubUWZ2TILp+vMSbaTrzEnNz6rQblbdeRltlct1O5RWDq9TjrtUYRuTj/Mq\nbOfcKjGU99OcVL3ZGe10V4mhtCz7uRlaN/IxkFU3K47Sysbaqi0OTZrynBrNnZPqfkvQADObDLwF\n+HDGuiWJ2V53721GTCIiE4WZzQfmj7qhJp2N3gr8fKzOUlX6uURDOhrS0ZCOpm19ajR3NuWmrZl9\nH7jG3b+dWu6um7aum7a6aStSj0ZzZ+4J38ymA48Be7r7c6l1Y57wRUS2deM24VftXAlfRKRujeZO\n/aetiEhBKOGLiBSEEr6ISEEo4YuIFIQSvohIQSjhi4gUhBK+iEhBKOGLiBSEEr6ISEEo4YuIFIQS\nvohIQSjhi4gUhBK+iEhBKOGLiBSEEr6ISEEo4YuIFIQSvohIQSjhi4gUhBK+iEhBKOGLiBSEEr6I\nSEEo4YuIFIQSvohIQeSa8M1sezP7oZndb2b3mdm8PPsTEZHKJuXc/peAq9397WY2CZiec39VmVkP\nlM4Oc6svBE6H7l3AgRdegLbfQgfgXfDcnjCzHQaBTQ7twKCF9X0e/p4BDBDOm+3AGqAb2EjYtf1A\nZ+z9eUL5vlh2MrABmBLnB+J8G8Pb3QBsJuy+/linDVgHdCXaL68HmApYrDsQ5ycDzwLbxzJ9wLRY\nZxqwNvY3I9HmFOCFuMxj/bbEdk2K2zuZsK8GifsollnL0NPucd362N/muKwttb4PmDYY+uqOK9fG\nOu2xj/ZYfr1Dl8V4N0HHozB5HbStgtW9MH0RTNkdnu+A6duFcmtWQfessIPWroLB4939WgAzWwjb\nnwdtu0P/GrB1MGUVrL6gXCYplC8tDnOrLwiPW+afhNJbYGMHtD8T2hqg3F6qbGb7ImPK3XOZgJnA\nwyOU8bz6z+irB7oclsapy2FKYr7kQ+sX+9ZlZ8dl5ccpcX15WbJOedkiH97fYfGxM2P97ArtJmM9\nLFGnUvudGcs7Ynu7ZaxblHickrE+q05bfEzul/S+Sm5v1rpyv6UK62dm9Nue6i/dfpfDjNR8sq9K\nddo3AQvD1NW/dTzl56OrD1iYOqYWhuVb2uqHzv7h+zdz/6TLDmtfk6ZKU6O5M8+ADgRuAy4B7gIu\nAjrHIujG4imtDC8sj9NShzmp+Xnx72N9eNljE49zUss8tX6pw9yMNnZI9Ju1Pqvd5LodEssrtT8v\nY3k54ZQy1s1NPM7JWJ9Vp5Top9q+2qHKurkVYj12hH6T/WW1Py9jfu4IdUoOpWVhympvS0zLUsdU\nhfJZ+zdrG9Nlt25fk6ZKU6O5c8QhHTM7Bvipuw/W+eZhEvBK4Ax3v8PMvgicA3w81f6SxGyvu/fW\n2Y+IyDbNzOYD80fdUA1nksuAh4HPAvvUcQbaCXgkMX844cQx6rNUg2dEDeloSKdCHQ3paJpYU6O5\n02Llkc4uM4HjgBMJd8ouAS539+dGqHcDcIq7/z5eyU9z9w8n1ru7Ww3npTGhm7a6aRvK6aatTGyN\n5s6aEn7sYDZwAvCPwH3AXsCX3f3LVeocAFxMyAYPASe5+9rRBi0iUmS5JXwzeyvhyn4v4FJgqbs/\nY2adwH3uvkf94W5pWwlfRKROjebOWj6HfyzwBXe/IbnQ3fvM7JR6OxQRkdao5T9tn04nezP7DIC7\n/08uUYmIyJirJeEfmbHs6LEORERE8lVxSMfM3g+cDsw1s3sTq7YDbs47MBERGVsVb9rGj2J2A+cD\nHyZ8xg/gOXdfNSad66atiEjdxvxTOmbW5e7rzGwW4QPPW3H31fWHOawPJXwRkTrlkfB/5u5vNrNH\nyU74e9Yd5fA+lPBFROqU+z9e5UEJX0SkfmP+OXwze2W1iu5+V72diYhI61Qb0uklYyinzN0XjLpz\nXeGLiNRNQzoiIgWR51crYGb7A/sSvnIRAHe/tN7ORESkdWr5AZQlwBHAfsDPgDcBNxG+SE1ERCaI\nWr5a4e3AG4Cn3P0k4ACGvlBdREQmiFoS/gZ33wxsiv99+wywa75hiYjIWKtlDP8OM+sm/Aj5nYSf\nK/rfXKMSEZExV9endMxsT2A7d79nTDrXp3REROqW5y9eHUH2VyvckFG8vs6V8EVE6pZnwv8pQwl/\nKnAosNzdX1d3lMPbVsIXEalTbp/Dd/e/TnW0K/ClejsSEZHWquVTOml/JPwTloiITCC1/OPVVxKz\nbcCBwPLcIhIRkVzU8rHMBxLlVgHfc3f9xKGIyART7euRJwOfBd4DPBoX7wR8BbjZzA509xUjdRB/\nQGUdsBkYcPdDRxmziIg0oNoV/gXANGB3d38Ows8eAheY2deBo4A9aujDgflj8ZOIY83MeqB0Nvhk\n6Hsapq2Dgbi2A2hbBat7oTQf1u8JnTuCbYTVF4YypbNhcDJsfBo27ADbd4XlfYPwwqXAztB9JDxr\nQ99GsR6YQtgtHXF+OrAWmEno34AX4vIBhm61DMY6fUBnfITwNE4C+uNyj/UH4zyEc24MjzVAu0OX\nwQbCh6+eS6xfG2Mozz8fY9kYY19D+LnjclvTY/99wIzYd0eMuy8Rc1eMbXOMrT3WG0is3whMjuU2\nxtg2xHimAps2QV8bTGsL9crb1xe3yQahfRNsngQd7bGdQZjaBgMO/ath+hpo7w7b1bk9bGoP7W+J\npQ+6NsMmh40DMD0+eeufho4/w5TUcTG4O8zoCBu25hdQip+eWN0L0xfBlN1h8DF4ticsLy2GwVmw\nsQumdsMLj8H6Hne/FsDMFoYyAKsvKC9vVD3tjUXfQ68rgNUXuvt59UctuXD3zAl4CGjLWN5OeJW/\nulLdVPlHgFkV1nktbeQxAT3Q5bA0Tl0OixxKPnz5Yallnan5jow6U2K5F2WsW+Qw22FxnC+XWRyX\nL0rNl+sm65TLdDh0J+bLZbtj/+l+k/GV67yshu05LPaftT2W2EfpmLsqtDczNT/S9qbrV9rn3VXa\nKVXYJ6VU2c64T9LPc1dctihRt9K+WhzLbrUvNkJn//C4ZntYzsIwdfUl6vQBC0dxnNfc3lj0nf26\noqdVr/NtdWo0d1Zr8PeNrMso+zBwN+FrGU4di6DHZoeVVoYD0uO01GFufJyXWr6Db112Xmp+tg9v\na04sV8pYV+7n2PhYSs2n1yfrpsvMTs0ny6a3Y24qvnKdWTVsT3kfZG1PKbE+K+Z0e+n9N69K3fKy\n9LZkxTi7gXYq7bdZcR9lPa9zE3Ur7atjM7Yzq/9kXKVlYRq2f5eN4jivub2x6Dv7dVVa2arX+bY6\nNZo7qw3p3G9m73X3bycXmtkJwP11vIk4zN2fMrMXAdeZ2QPufmOivSWJsr3u3ltH2yIi2zwzmw/M\nH3VDVc4gc4DbgV8BF8bpV8AdwJwGz0qfABaP9iw1RmdIDeloSMc1pNNY2fpeVxrSySF/eSP1qn61\ngpkZ8DrCj584cJ+7/6LWk4mZdQLt7v6cmU0HlgHnuvuyuN69hV+toJu2ummrm7aNl63Shm7a5mxc\n/qZt/HbNH8fZScBl7v7pxPqWJnwRkYloXCb8ETtXwhcRqVujubOR79IREZEJSAlfRKQglPBFRApC\nCV9EpCCU8EVECkIJX0SkIJTwRUQKQglfRKQglPBFRApCCV9EpCCU8EVECkIJX0SkIJTwRUQKQglf\nRKQglPBFRApCCV9EpCCU8EVECkIJX0SkIJTwRUQKQglfRKQglPBFRApCCV9EpCCU8EVECiL3hG9m\n7WZ2t5ldlXdfIiJSWTOu8M8E7gO8CX2NOTNbaDZrWZhs4Vi2ldX20LIZy826lw89jtz/UN2pD5qV\n1pltt9Zs+wdj3Z5U3z1ms1amylSKKZYtrTebtj78bT1btzHzSbPudWYz1oe+ZyxPxhvKltaZbbfZ\nbOZms9Ims+kPZm1T9f1S3iflbZqxPMTQvbx6W93LQ3+zVpZj23pfb7/VuqG63csTy9P7cMTnMxVL\n3F9b9t+Ix1b1fTH8ea12fIjg7rlNwBzgf4AFwFUZ6z3P/scg/oXQ1QdLPUxdfcDCMWqrHzr7U233\nhMfFDrN96HHk/ofaX+TQlVU3Llvq0LUROrPKZMV0SaibbGeRh/pZ/STj7uwPcdFTuWznxuQ2Vdjn\nPVsvm52IIR1/tba2im1jiC8zpv7YZ2JflGJ/5fnO/q3XZ+67ZCw9W+/Hzvg8VH5ua9sXWz2vDR+f\nmibW1GjuzDuo/wJeARwxMRN+aVl4IXmcljqUlo1dW/PSba8Mj8f61o8j9z/U/twqdY9N9Z1VJh1T\n98bhZebGctX6KbdVWjZ8u9L9DW1ThX2+snoM2fsnu61kbNX2QVafyX2T1few5zMZS6q96rHXty+O\nrdiGpm1zajR3ThrTtwsJZvbXwDPufreZza9Sbklittfde/OKSURkIoo5dP6oG8rxDHQe8ATwCPAU\nsB64dCzOUk08i2pIR0M65Stn15COpvEyNZo7LVbOlZkdAfyTu78ltdzd3XIPYBTCjbDS4jC3+gJ3\nv3as2gqPW7c9VOaFWdABDBAe21aN1P9Q3fV7QueOMODQ/gy0PwKre6E0P9H3QVA6GzZ2JMpUiqkn\nlPVpsAHo3ACrL4xlYxttz0PbdjDQDpM3w8YHYX1POd7QRvc5MDA9fFag3eGFR6DvjPQ2Ze3zoWWD\ns8I+mbIqbNP0RTBldxh8DJ7tqdzW4CzY2AVTu+GFx2B9z9C2vjALJnVB+5Z1Q31ufx607R6XX5na\nh4z0fKZiifsR4v5bPtKxVX1fQPp5Hc3xKRNHo7mzmQl/sbsfk1o+7hO+iMh4M64TfsXOlfBFROrW\naO7Uf9qKiBSEEr6ISEEo4YuIFIQSvohIQSjhi4gUhBK+iEhBKOGLiBSEEr6ISEEo4YuIFIQSvohI\nQSjhi4gUhBK+iEhBKOGLiBSEEr6ISEEo4YuIFIQSvohIQSjhi4gUhBK+iEhBKOGLiBSEEr6ISEEo\n4YuIFIQSvohIQSjhi4gURK4J38ymmtltZrbCzH5jZkvy7E9ERCrLNeG7ez+wwN0PBA4EjjKzV+XZ\nZ5GY2UKzWcvCZAub3U+z+q+3v3riGuttqNZes/fXeOl7oijEPnL3pkxAJ7AcOCSxzJvV/7Y2AQuh\nqw+WepjfqjpLAAAJhklEQVS6+oCFzeqnWf3Xu731xDXW21CtvWbvr1YcKxN5mmj7qNHc2YzA2oAV\nwHPAp8ciaE0OlJaFA9PjtNShtKxZ/TSr/3q3t564xnobqrXX7P3VimNlIk8TbR81mjsnjeGbhUzu\nPggcaGYzgR+b2X7u/tvy+tS4fq+79+Ydk4jIRGJm84H5o26oyWeljwGLR3uW0qQhHQ3pjL9jZSJP\nE20fNZo78w5qNrB9/HsacANw9GiD1rRl/y0cGl7JO9kO76dZ/dfbXz1xjfU2VGuv2ftrvPQ9UaaJ\ntI8azZ0WK+fCzPYHvg20E8byr3D3TybWu7tbbgGIiGyDGs2duSb8ETtXwhcRqVujuVP/aSsiUhBK\n+CIiBaGELyJSEEr4IiIFoYQvIlIQSvgiIgWhhC8iUhBK+CIiBaGELyJSEEr4IiIFoYQvIlIQSvgi\nIgWhhC8iUhBK+CIiBaGELyJSEEr4IiIFoYQvIlIQSvgiIgWhhC8iUhBK+CIiBaGELyJSEEr4IiIF\nkWvCN7Ndzex6M/utmf3GzD6UZ38iIlJZ3lf4A8BZ7r4fMA/4gJntm3Ofo2Zm81sdQ5piqo1iqt14\njEsx5SvXhO/uf3b3FfHv54H7gZ3z7HOMzG91ABnmtzqADPNbHUCG+a0OIMP8VgdQwfxWB5BhfqsD\nyDC/1QGMlaaN4ZvZHsArgNua1aeIiAxpSsI3sxnAD4Ez45W+iIg0mbl7vh2YdQA/Ba5x9y+m1uXb\nuYjINsrdrd46uSZ8MzPg28Aqdz8rt45ERGREeSf8w4EbgHuAckcfcfef59apiIhkyn1IR0RExodm\n3bQ9ysweMLMHzezDFcp8Oa7/tZm9YjzEZWbHx3juMbObzewvWx1TotwhZrbJzI4dDzGZ2Xwzuzv+\ng11vq2Mys5lmdpWZrYgxnZhzPN8ys6fN7N4qZVpxjFeNq0XH+Ij7KpZr5jFey/PX7GN8pOeu/mPc\n3XOdgHbgD8AeQAewAtg3VeZo4Or496uAW8dJXK8GZsa/j8o7rlpiSpT7JeFm+KJWxwRsD/wWmBPn\nZ4+DmHqAT5fjAVYBk3KM6TWEjx3fW2F904/xGuNq6jFeS0yJ57gpx3iN+6mpx3iNMdV9jDfjCv9Q\n4A/u/qi7DwDfB96aKnMM4eYu7n4bsL2Z7djquNz9FndfG2dvA+a0Oqbog4SPuf5fzvHUGtPfAVe6\n+x8B3H3lOIhpEOiKf3cRPjiwKa+A3P1GYE2VIq04xkeMqwXHeC37Cpp7jNcSU7OP8VpiqvsYb0bC\n3wV4IjH/x7hspDJ5H3i1xJX098DVuUZUQ0xmtgshuf1HXJT3TZha9tNeQCl+b9KdZnbCOIjpq8DL\nzOxPwK+BM3OOaSStOMbr1YxjfEQtOMZr0exjvBZ1H+OTcg+p9icr/ZnSvJ/kmts3swXAycBh+YUD\n1BbTF4Fz3N3jx17r/ixuDjF1AK8EXg90AreY2a3u/mALYzoKuMvdF5jZXOA6MzvA3Z/LKaZaNPsY\nr1kTj/FaNPsYr0Wzj/Fa1H2MNyPhPwnsmpjflXB1U63MnLis1XERb2JdBBzl7iO9DW1GTAcB3w+v\nA2YDbzKzAXf/SQtjegJY6e4bgA1mdgNwAJDXi6GWmE4EPg3g7g+Z2SPA3sCdOcU0klYc4zVp8jFe\ni2Yf47Vo9jFeixOp8xhvxpDOncBeZraHmU0G3gmkn7ifAO8BMLN5wLPu/nSr4zKz3YAfAe929z/k\nHE9NMbn7S9x9T3ffkzDG+f6cXwi1PH//DRxuZu1m1km4KXlfi2N6HHgDQBwr3xt4OMeYRtKKY3xE\nLTjGR9SCY7wWzT7Ga1H3MZ77Fb67bzKzM4BrCXfe/9Pd7zez0+L6b7j71WZ2tJn9AVgPnDQe4gI+\nDnQD/xGvNgbc/dAWx9RUNT5/D5jZzwn/YDcIXOTuub0YatxP/wYsNbN7CEMC/+Luq/OKycwuB44A\nZpvZE8AnCMMALTvGa4mLJh/jNcbUdDU8f009xmuJiQaOcf3jlYhIQegnDkVECkIJX0SkIJTwRUQK\nQglfRKQglPBFRApCCV9EpCCU8GVcMLNfmtkbU8v+0cz+vUqdR82sNEK7Pan5m+PjHiN9PW+F9t5m\nZoNmtncNZS8ys30zlp9oZl+Jf59W/l6WuPzF9cYkUislfBkvLgfelVr2TuB7VerU8k8kH9mqgvto\nvyvmOODG+FiVu5/q7vePUOYb7v6dOPteYOdRxidSkRK+jBdXAm82s0kQrsCBnd39JjM7zsIPdNxr\nZudnVTazH8dvMfyNmZ0al50PTLPwoxXficuez6jbbmafM7PbLfwYyPsq9DGD8OVip5A4OcX6n4/x\n/drMPhCX95rZQfHvk8zsd2Z2G/BXibpLzGyxmS0CDgYui/EebWY/TpQ70sx+VPPeFMmghC/jQvyX\n8NsJPxQCIaFeYWY7A+cDC4ADgUPMLOs3Ak5294OBQ4APmVm3u58DbHD3V7h7+etss94V/D3hu20O\nJXzX/qnxhJP2VuCa+A2Jq8zslXH5+4DdgAPc/QCG3pU44HGYZgkh0R8OvCwRh4fN9ysJ3xH0dzHe\nq4F9zGxWLHcS8J8ZMYnUTAlfxpPksM474/whQK+7r3L3zcBlwGsz6p5pZiuAWwjfSrlXHf2+EXiP\nmd0N3AqUgL/IKHcccEX8+wqGhnVeD3zD3QcBUt84aYQv2ipvw0CsW+krf5PLvwOcYGbbA/OAa+rY\nJpFhmvH1yCK1+gnwBQu/99rp7neb2a6pMkbqKt3M5hOS7jx37zez64GpdfZ9hrtfV2llvDm8AHi5\nmTnhS9sGgX9OxFVJ+l1FrWUvAa4C+oEflE8oIo3SFb6MG+7+PHA9IdGVh0XuAI4ws1lm1k54B/Cr\nVNUuYE1M9vsQrobLBsr3Baq4Fjg9cf/gpfErcJPeDlzq7nvEr+7dDXjUzF4DXAecFuPDzLqTm0X4\n6cAjzKxkZh3AOxhK7Mkf+HiOoZ+sw92fAv4EfDTuE5FRUcKX8eZyYP/4WE565xBOBCuAO939qli2\nnDR/Dkwys/sIPwhxS6K9bwL3lG/asvUVdPnviwnfbX5X/KjmfzD83e+7gB+nll0Zl19M+G7ye+Kw\n0laf4HH3PxPG8G8BbiL8GHYyhnIcS4Gvm9ldZjYlLvse8Li7/w6RUdLXI4uMY2b2VWC5u+sKX0ZN\nCV9knDKz5YRhniPjzV6RUVHCFxEpCI3hi4gUhBK+iEhBKOGLiBSEEr6ISEEo4YuIFIQSvohIQfx/\n11M2tnQTUHUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8312780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(df['volatile acidity'], df['quality'])\n",
    "plt.xlabel('Volatile Acidity')\n",
    "plt.ylabel('Quality')\n",
    "plt.title('挥发性酸度（volatile acidity）与品质（ Quality）',fontproperties=font)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "这些图都可以显示出响应变量与解释变量的相关性；让我们构建一个多元线性相关模型表述这些相关性。如何决定哪个变量应该在模型中？哪个可以不在？通过`Dataframe.corr()`计算两两的关联矩阵（correlation matrix）。关联矩阵进一步确认了酒精度与品质的正相关性，挥发性酸度与品质的负相关性。挥发性酸度越高，酒喝着就越向醋。总之，我们就假设好酒应该具有酒精度高、挥发性酸度低的特点，虽然这和品酒师的味觉可能不太一致。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "###拟合与评估模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "现在，我们把数据分成训练集和测试集，训练回归模型然后评估预测结果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R-squared: 0.382777530798\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pylab as plt\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.cross_validation import train_test_split\n",
    "df = pd.read_csv('mlslpic/winequality-red.csv', sep=';')\n",
    "X = df[list(df.columns)[:-1]]\n",
    "y = df['quality']\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y)\n",
    "regressor = LinearRegression()\n",
    "regressor.fit(X_train, y_train)\n",
    "y_predictions = regressor.predict(X_test)\n",
    "print('R-squared:', regressor.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "开始和前面类似，加载数据，然后通过`train_test_split`把数据集分成训练集和测试集。两个分区的数据比例都可以通过参数设置。默认情况下，25%的数据被分配给测试集。最后，我们训练模型并用测试集测试。\n",
    "\n",
    "R方值0.38表明38%的测试集数据都通过了测试。如果剩下的72%的数据被分到训练集，那效果就不一样了。我们可以用交叉检验的方法来实现一个更完善的效果评估。上一章我们介绍过这类方法，可以用来减少不同训练和测试数据集带来的变化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.290041628842 [ 0.13200871  0.31858135  0.34955348  0.369145    0.2809196 ]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.cross_validation import cross_val_score\n",
    "regressor = LinearRegression()\n",
    "scores = cross_val_score(regressor, X, y, cv=5)\n",
    "print(scores.mean(), scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里`cross_val_score`函数可以帮助我们轻松实现交叉检验功能。`cv`参数将数据集分成了5份。每个分区都会轮流作为测试集使用。`cross_val_score`函数返回模拟器`score`方法的结果。R方结果是在0.13到0.37之间，均值0.29，是模拟器模拟出的结果，相比单个训练/测试集的效果要好。\n",
    "\n",
    "让我们看看一些模型的预测品质与实际品质的图象显示结果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEaCAYAAADzDTuZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYpGV57/Hvr6d79hmYnsAogkoENLgcgQRGFukoixDE\nNTmIypYgYrCRYTBGOQFPxKNHWQQXcAQyJrhwUBQSlAF1WCKiArKjQtCwyCINzD7T032fP5636erq\n6umqme566+n+fa6rrq73qaq37oKp965nV0RgZmaTW1vZAZiZWfmcDMzMzMnAzMycDMzMDCcDMzPD\nycDMzID2sgMwq4ekDmBKVXE/oIhYP8bvJaADmAnMArYC5gHbAvdHxAObed5tgd6IeHasYh3l/cbl\nc9jE5GRgLUnSRcCuwFRga+A/gNcCOwHPAM8CUTz3cuA04ClgPvBZYAXQAyyMiM9K+iBwSvGcSi8F\nPhAR1xbn+i3wB2BdcVsLrCzerwd4siLG2yJijwY+1l8CbwY+UPVZjwH+V/F5rgD2r/Ha4yLi/jH8\nHE80ELdNAk4G1qpOAQ4ADgO+GxE/BJD0TWBxRDxWHL8F2B04i3SBmwPMBo4Czqs4XwBnRcTXK99E\n0hlV7/vHiHhjxeP/CCyLiNuqXjcF2FnSLSPEfydwN3Ai6SI8oKPiNbOAy4q4/zfwauC6iPjYCOcc\n889hNsDJwFpSRKyR9HagPSJ+KOko4FhSP9cXJW0D/AC4iHRRnU1KIF/bxGlVT5mk84HdisOXA0dI\nWlEcXx4RFwAvAW6MiMNGfDPp74BTgRcDD0bEzZIOBB6PiHslvbHifQD2AU7fRPzj8TnMACcDa1FF\n08/2wHaSbgJ+AnyedOF/HHgFsIDUrPKlOk7ZC3y0uEBXWgDcWHEcEdFdEcfngO9FxH9WvW4bYKGk\nX4zwfvtExNcktZOars6V9Pridb8E7o2IG4EbJR1NSmb9wCJJbyvOsStwX3H/tog4aRw+hxngZGAt\nKiL+RtJfkfoI9gP+WDz0ImBVcf8h4CDgrcVxdQdz5fkuAS6p570lvQL4KfCbomhh6oulMyJeXZzv\nNuBPRnj9bRGxoTj8a+D9wBnA7cDnSP0Z1VYBdwE3AUtITUs/ioj9JM2NiBXj8TnMBnhoqeXgBmBg\nxNB2wPMVj11AGi0zhfTvedjKi5LeK+k3km6S9GtJR0s6XdJdRdnviiabStdExH7AvwDHF/d7Gg08\nIr5Jaip6TUT0ATsCj4zw9EuBdwBfAHYpYt+e1K+ApPcV8Tf9c9jE55qBtbyIuEDSCaR/r+8ELibV\nGAD+HPg4cAgpKfwC2AC8reo0n4qIrxdNMpCSxkkRcaOkT5BGYm7L8FE6uxXnfIGkK0lNWMNCBT5J\n0X4v6Uzg4IrXnUbqP7is+IUOqbbwc+D1pKavrYCtI+IeSUTEo5Iek7QPFZ3HY/E5zCo5GVgrm8rg\nL/1vAV8FriF1Gl8O/A64lTSW/jTgG8C3geOAd5GGcQ68fqqk6cU5B5pwVPX3DcAdpF/OV0maT+rU\n3U7SOcBSgIh4x0gBS3olxbDNiDgTOLMo35XUuf08cDVwUUQ8Uzx2NKnm8wCp5vOZ4nQdxd/PAP8M\n/HAsP4dZpaY0E0l6paQ7Km7PS+quek5XUT7wnHpGVdgEJeliUvv6nZL2InWOXh0RHwHeArwS+C5p\n+Oa1wKKI+CxpxNEhwHdISeOO4pSnkkYfncJggjhb0nWkYagAxxfn2pfUT/F94ATgJODdwJGS3lwj\n1k9KekDSncC/k5p5kPSnko6VdBVwLmmY6T7Aw8A1ki6UtEtxmvsi4hvAh4CdJN1F0ZwTEb+LiPeP\n9+ewyU3N3txGUhvwGLBnRDxSUd5F+kIf3tSArOUVM2mnVs80lqSICEkzImLtGLzPXsA9wN8DN0fE\nT6sefynw0oi4uc7zHQnsAFwZEb+p8fjbgHUDE8XGylh/DpscykgGBwH/FBH7VpV3AadGxFtrvtDM\nzMZNGaOJjiC17VYLYG9Jd0q6pmhjNTOzJmhqzUDSVFIT0a4R8XTVY3OAvmLm6SHAFyJil1rnMTOz\nsdXsZPA24MSIeEsdz30Y2CMieirKmtumZWY2QURErWVMXtDsoaXvAb5Z6wFJC4Cnig7BPUmJatjk\nmNE+UCuTdGYx3DBLOcefc+zg+Ms2AeIf9Yd005KBpFmkVSiPryg7ASAiLiINeTtR0kZgDalvwczM\nmqBpySAiVlO1lkuRBAbuf4n6FhwzM7Mx5rWJmmt52QFsoeVlB7AFlpcdwBZaXnYAW2h52QFsoeVl\nBzDemj7PYEsMzDEqOw4zs5zUc+10zcDMzJwMzMzMycDMzHAyMDMznAzMzAwnAzMzw8nAzMxwMjAz\nM5wMzMwMJwMzM8PJwMzMcDIwMzOcDMzMDCcDMzPDycDMzHAyMDMznAzMzAwnAzMzw8nAzMxwMjAz\nM5wMzMyMJiUDSa+UdEfF7XlJ3TWed76k30q6U9JuzYjNzMygvRlvEhG/BnYDkNQGPAZcWfkcSYcC\nO0XEzpL2Ar4CLGxGfGZmk10ZzUQHAA9FxCNV5YcDSwEi4lZga0kLmh2cmdlkVEYyOAL4Ro3ylwCV\nCeJRYPumRGRmNsk1pZlogKSpwFuBfxjpKVXHUeMcZ1YcLo+I5WMSnJnZBCGpC+hq5DVNTQbAIcBt\nEfF0jcceA3aoON6+KBsiIs4cn9DMzCaG4kfy8oFjSWeM9ppmNxO9B/jmCI9dBRwFIGkh8FxEPNms\nwMzMJjNFDGuJGZ83kmYBvwd2jIiVRdkJABFxUXH8ReAtwGrg2Ii4veocERHVTUlmZrYJ9Vw7m5YM\nxoKTgVleJB0Mnaemo56zI+LaciOanJwMzKw0KRHMvRLOn5FKutfCinc4ITRfPdfOZncgm9mk0Xkq\nnDMDjh4omAGLTgWcDFqQ1yYyMzPXDMxsvPScDd37ApXNRGeXGpKNyH0GZjZu3IHcGtyBbGZmdV07\n3WdgZmZOBmZm5mRgZmY4GZi1NEkHS/OXpZsOLjsem7jcgWzWojyD18aKO5DNstZ5akoER5Nu588Y\nHKaZB9ds8uFJZ2Y2LgZrNucM1Gz2leSaTYtyM5FZi8q9mUiavwyOPRAeLkp2BC69LuKZg8qMazLy\nQnVmGYuIayV9ChYtSiUrzsklESTr58NS4PPF8eKizFqRk4FZiypqBqdXNLOcLum2fBJCBykRHF1R\n9pGSYrHROBmYtazcl4Bue6a+MmsFTgZmNk68amlO3IFs1qJy70AGkHQtzDswHT17XUR4eGkJvGqp\nWeZyXgJa0qUw9xg4vyjpBlb8S0QcW2JYDcn5v38lJwMzK43UuRHOnTLY57EUOKUvoieL5umJUDMb\n4BnIZpnLfAZvG1wN7FTcri7KcpH/DPBGZJGhzSaj/GfwruiD69qHNhOt7iszosbdDbyruL9jmYGM\nu6ZlaUlbS7pC0v2S7pO0sOrxLknPS7qjuJ3erNjMWlPuv0znKiWCF+IvynLRsxyWAIcXtyVF2cTU\nzJrBF4BrIuLdktqBWTWec0NEHN7EmMxaXNa/TNcAc2qUZaKzC85h6KS5RV3Ap0sJZ5w1pWYgaStg\nv4i4BCAiNkbE87We2ox4zPKQ+y/TZ2+BDwFvKG4fKsqsFTWrmWhH4GlJl0q6XdISSTOrnhPA3pLu\nlHSNpF2bFJtZi+rsguOBq4rb8UVZLmYvTEtSfLC4dRRlueg5O40gWkq6da9NZRNTs5qJ2oHdgZMi\n4heSzgM+BvxTxXNuB3aIiDWSDgG+B+xSfSJJZ1YcLo+I5eMWtVmpcl/orW12ah2ubGbpnl1WNI0q\nFgp8R7EECLAim3kGkrqArkZe06xk8CjwaET8oji+gpQMXhARKyvu/0DSlyV1RkRP1fPOHO9gzVpD\n7gu9TanR8lCrrHUVF/8sEkCl4kfy8oFjSWeM9pqmJIOIeELSI5J2iYjfAAcA91Y+R9IC4KmICEl7\nkibE9dQ6n9nkkXMHci+pNjNgcVFmraiZo4k+DFwmaSrwEHCcpBMAIuIi4N3AiZI2kkYcHNHE2Mxa\n0Nq5qdO4cpz++rklBtSg6IUNHXBhcbyhKMvHRFmOoh5ejsKsRUnz/wjnzB+6nMOiZyKe+ZMy46qX\nNP23wE6wTVHyNMCDEet2Li+q+k225Sg8A9msZfU9C8yvUZaRKcD2xf0eIKcJyLnvJ9EYJwOb0PKu\n5ufe5t6xHUwnDSuFFP+67UoMyDbBycAmrPzX9pk+L/0qvao4Phq4dF6JATWoY9rw0VCnTCsrmsZN\nrs15nAxsAsu9mr/+97BkPgzMv7we6Pt9mRE1pn893D1z6Gio/vVlRtSInOcZbI6sxvyaTS6rv5P+\nDszgrSzLwbrHhy+nse7xcmOykbhmYBNY7tX8zi44lsFmouOBS7vIZqG0GSvgPKomza0oK5pG5d/M\n2BgnA5uw8q/mr9sRvgQMjCS9BmjLbeZZxnJvZmyMk4FNaLkuJ5D0vQymAZ8qjruB9S8rMaAG5T4a\nanJxMjBrWTM74FyqRuN0lBVN4zoYPhrq4vLCaVjuzYyNcTIwa1m1VgfIZ8WAVAu4lLRBDMAicqoZ\n5N/M2BgvR2HWoqRZfdDeNji09D5gY3/E6illxlUvafZtELtDZ1HSA+j2iFV7lBnXZFTPtdNDS81a\n1xNpOYeBoaVTirJcxFyYSerz+BTpfmS00F4aUSTNX5ZuOrjseMaTm4nMWlb7tOF9Bt0ZzeCdPm/4\nDORF2cygnmxDS10zMGtZUzrrK2tV/b+Hq4GditvVRVkuOk9NK5YeTbqdP2NwnauJxzUDs5bVqxpD\nMzPqM3vuLrhu96H7May4q8yIbGTuQDZrUfl3IM9bB+dNG7ofw0fWRzw7vcy46uX9DMysRbStg40z\nB483FmW5UI3+jVplrWmyDS11MrAJLe/9DNashekzh5flYgPDm7k2lBTL5sl7Bntj3ExkE1ZKBNP+\nHbYpfvQ8vRHWH5ZLQpC27oe/EzxclOwIfC0insti4Ic0ax20TYPXFCX3AP3rI1Zn0Uw0kXiegU1y\nUy+Bae2D49yntaeyXPQptbMPLAG9tCjLhe6tr8xagZuJbAKb9eIaa/u8uKxoGtfO8HH6Hykpls2x\n+jswc/fB435gTUb7MUwurhnYRNZfZ1mL6g+4m7RT2LtI9/vzadelswtOBLYrbicWZdaKXDOwCezZ\nf4XuYwaPu4EV/1pWNI3bELBEQ8fpb8goGQC8llS7gdTMZa2qaR3IkrYGvga8mrT04nER8bOq55wP\nHAKsAY6JiDuqHncHsjVE0qUw773p6NnLIuLYciOqn9S5Ec6dMnSc/il9ET1Z/IhLHfgzvw+vK4aT\n3rUe1rwtlw78iaTV5hl8AbgmIt4tqR2YVfmgpEOBnSJiZ0l7AV8BFjYxPpuAiot/NglgqL6NFKvT\nVZVlpK8dHq24b62qKTUDSVsBd0TEn27iORcCP4mIbxfHDwD7R8STFc9xzcAakvM8A2n2auiYmfYR\nhtR53LsmYtWsTb2uVUjTHoPp2w1t5lr3eMT6l5QZ12S0xTUDSa8C5kXELZI+HBEX1HjOm4H1EXHz\nJk61I/B0qrLzP4DbgJMjYk3Fc14CPFJx/CiwPfAkZpsh/1UnN04ZOnFrNdCWxVIUSe6juSaX0apt\nfwa8CrgF+BvgAkn3knapgNSIeSRwTB3vsztwUkT8QtJ5wMeAf6p6XnXmGlZtkXRmxeHyiFg+ynvb\npNV5Khw7Y3DbxeNnwKUZbWiujrTj4kAHbDewIaNtL+lnWDNXTqO58iWpC+hq5DWjJYOngL2rynoi\nYr/iDfcF7oqI/x7lPI8Cj0bEL4rjK0jJoNJjwA4Vx9sXZUNExJmjvJdZYf384dsurp9fYkANmtFW\n45d1RsPBVz8JJ2wHpxfHTwPhmn4TFD+Slw8cSzpjtNeM9g/rWWCepD2AuZL2KU78KkmvI327/m8d\ngT0BPCJpl6LoAKB6JuJVwFHF+RcCz1X2F5g1rn1u6ju+qrgdW5TlIvc9kPuBaVTMAMcVg9Y1Ws3g\neWAOcBBpI9O3F+X7ALuR+gK2Bx6v470+DFwmaSrwEHCcpBMAIuKiiLhG0qGSHiQ1jmY6AsRaR9+2\nsITBJaCvB/q3LTGgBq3sh+6KH2zdwOqMrqZzFtSo2SwoKxrbtNGSwWpgRkT8H0mHRsRpkm6KiIsB\nJM0Hvg/sO9obRcSdwF9UFV9U9ZyT6g/dmiXfETnRDlNJ+wdD6ohdm9HwxrmC46jo8wAuyWg0Xa2h\nihmtjDnJjPbFWANUrzC4SzEqCNK/0p9L2jsifjrm0Vnp8h6RM7Vv+No+p/SVFU3j+vvgte1DZ/D2\nZxT/hqdg8XaDx4uLMmtFoyWDXgaTwcBs4MoO5R7Sjhv7Ak4GE1LnqSkRvHBBnVFs9pFBMuirsXh+\nrbJW1dtWY9vLjDqQp98LR283WLM5GrjUq5a2qE0mg4gISS+WdAuApMoL/lOkBHEt8OPxC9Fsc/Vv\nlcY4DFhUlOVialu6gFZeTC/OKBn0nA1L9q3aNvLscmOykWz2DGRJ2wJvIDVk/nNE3DqWgY3wnp6B\n3GQ57wMrdW6A4zqGbg5zSW9Ez9Qy46qXND3SCJzKGbzriViXzXdA0sehs8jIPedExKfLjWhyqufa\nuVnJQNIbqZggFhE3NB5e45wMypFrB7Kka2HuQUMvpiuWRcTBZcZVL6mzH94k+FVR8nrgxxHRk0Xt\nwAvVtY4xSQbpf+gLeiPix5J+RxqzJ+CoiNil5ovHmJOBNUKa9Vto26lq28UHI1bvXGZc9ZJmb4AZ\nHYMdyIuBtb0RqzKp2cy+DabsPji09z6g7/aIVXuUGddkNFarln4VuIR04T8S2AV4JCLOKt5k/y0N\n1Gx8THtZGpo50Ey0D3DJy0oMqEHtvfDKDvhocfxK4J7eMiNqjHaqr8xaQT3J4L8j4pPwwqJ0AFtV\nNBXNG6/gzLbMhl5YWvXLekNGF9M1U9IYjdcVx3cAvRktVEeNeR7rMprnMbls7v+Yy4H9ivtXjlEs\nZmNsylr4/Myh8wy615YWTsOmTU0ju4dcTLNoIko6Mp/nMblsbjKoXGraMwqtRU2psQ5RrbJW1aEa\nF9OM+szit6TViqvLrAVt7qiEjxd/pzJ89VGzFtG7Ns0tWFrcFhVluai1qVlOG50993H4UG8agf4G\n0v3nPj7aq6wc9dQMtpV0FKl/oLMoeyoilkqaTupUNmtBq34G0w4aXEJ5NbD+Z5t6RWvpA06uOD65\nKMtJe9tgM1d3FkNiJ6t6ksG5pB02AAZmDy4oEsRU4EXjEZjZlpu9EDpIyycDnAJ0ZLSv9kbSxX8g\nma0jr1bZrb4IX5hS0cw1BU7+IpDF0N7JZtRkEBEX1ij+ZPF3LYPfNLMWM2XW8CWUT85i/+BkKqkD\neeArtpiUEHLRVmMYb60yawWjJgNJ/7CJh4PBBezMWoxqdLbWKmtVHdToQC4pls3RuxYWV2zTuZi8\n+mwml3qaiTbVxtpB2pPgFWMTjtlY2tgPiyvaqRcXZbnIvQN51Weh/ywYaFxYA6z5bJkR2cjqSQad\nFNtRVgngXcCdYxqR2ZjRBji6feiqn1/NaAnrYGhN4BRy6jOIiE9LAn5TLFS3xgvVtbB61iY6eoSH\nfkiafXxIRJw71oGNEIvXJrK6SXP6YbqGzkBeFxErsxjVIs0L+FuGrrp6MRHP+jtgDann2lnPl+If\ngFVVt5Wk3q1LgGu2ME5rcZIOluYvSzdlseJn0q7B/QCuIt1vz+hC+tzGtB7k4cVtSVFmNvbqaSZ6\nhnTxH6ivTiElkZnA9yPi1+MUm7WAvLe97O2HpW1Dawa9GfUZzGuvsQey1/axcVHXpDPSXIJvVZRN\nATYAp0n6ZUT8aDyCs1aQ87aXHavh6DlVO4WtLjOixgTwWobugZxPnwHkuxfGZFRPMvgcsD3wa1KN\noJ9UK/gBcD1wraSFEeEhY9ZinrsFllRvbnNLmRE1Zm1Ad0WzVjewPptskHetcvLZZAeypGnA1cA5\nwC+BK4CDgAuAPwduBe6OiK+Mf6juQC5D3tteznkemDt0cxtWRKzMYh9kaU4f9LcNrgLTA7T1R6zM\nYhlraf4yOOfAwVrlUmDRdRHPHFRmXJPRWGxuEwAR8UNJnyB1Fr8VWFbc/gh8V9KFMcqwpGJ3tBWk\n+fW9EbFn1eNdwPeB/yqKvhMRnt1csoi4VtI7iqYhYEVGVf0ps+ALDL0Y5TQDOUiV8e2L4+fIrZnI\n8jFaMmhjcK/jQ0jDGWZXPL4N8PXREkEhgK6I6NnEc26IiMPrOJc1UXHxzyQBVKo1WC6LUaUD2obG\n2wb0ZfQBes6G7n15YW2z7rWw4uxNvsRKU0+fwTxJLwGeBh4C5gLPA0+RZh+/pYH3G62Jx01ANoZ6\nV8Piiv0LFhdluRA1dgorL5wG5V2rnHzqSQbzScMZOoHtSCOLtgPeCVwTEfXO6Azgekl9wEURsaTG\n43tLuhN4DFgcEffVeW6zGlZ9F/qPqVoO4bslBtSg3NcmyrlWOfnUkwwejIj3SPoJaUH4h4rbLcCR\nkt4XEf9Wx3n2iYg/SNoGuE7SAxFxU8XjtwM7RMQaSYcA3wN2qT6JpDMrDpdHxPI63tsmpVmvSyOg\nf1Mc9xZluag1JSKjaRJWmqIPtquh14wymqgD+GxELCr2L9iJ1Gm8vnjKt4AbgT0jYv0Ip6l13jOA\nVRExYvuhpIeBPSr7GDyaqBySPg6dxfoyPdmsLyPNWA1TZw4dWrphTcTaLDqRpamRmtsr419LxIZs\nvgOeZ9Aa6rl21rM20a+Bk4sRRTcBT5I6ei8oHt8L+PmmOpElzQSmRMRKSbNII5E+GRHLKp6zgLSD\nWkjaE7g8Il7e6AeysZUSwdyzqsbqfyKHhCB1boTjpgxd2+eSvoieLGbxprWJ3gz8qih5PfCjbNYm\nynlY8kQzFkNLIY1ne6ekI4EdSHMN2iSdTGrnv7OO0UQLgCvTCoa0A5dFxDJJJwBExEXAu4ETJW0k\nNe4eUUdsNu46F6VpJpXt1osWAS2fDKBXaQDcrsXx9UB/FhfSRMGwQRXKaGxpzrPXJ596ksEG4N+B\ntwOPAw9WPDYVuJjUfDSiiHiY9LOmuvyiivtfAr5URzxmdeon7RRWORpnTXnhNOzZgOtUVSvLKBlY\nTupJBvcB10fEVZIuj4j/qHxQ0mHjE5q1hp5zoPusweNuYMU5pYXTkKltw0fjfCSjcfrz2oZv23lK\nRvF7nkFORu0zaCXuMyiHpEth3nvT0bOXRcSx5UZUn9z3A5A6Y2gyWAqcQkRPFvGDO5BbxZh0ILcS\nJ4Pmy7kTMPfRONL0gGkMjX89EeuyiN9ax1h1INuklnMn4EyGrk0EcHJJsWyOmcCbgH8ujg8Eflxe\nODahORnYKPrn11fWinJfm6gXuIGqzXnKC8cmNCcDG0Uv6SI0IKcLUi+paWVgOYr7yGsG78Y+WDNl\n6HIa/X1lRmQTV04/k6wU054Zvo/wtGfKjaleA5PiP8jg8NK6J8q3gBlrYDfSqu7/Rbo/I6exsRnv\nnz35uGZgo+g5G5bsW9WBnMnwwNnUGJpZUiyb47mAuxnagbwymxEf3uksL04GtklehrhMW8+pkczm\nlBVN43IefDD5OBnYqPJdhvj5fuiuaArtBlbl1GlQayigh5XauPA8A5uwpDn9sFYwsOXx88CMiFiZ\nRV9Z/vMk8p2jMtF40plNatKcSKOHhmwoT8TKLP4NpRnUrwZ+W5TsDNybzQxq8AzkVuFk0CL8hSiH\nNLP4ZT2wlNIi0i/rNVn8G5Jm9cHMtqHzDNb0R6yeUmZclh/PQG4BE2FERb7JbBrDl9/+SEmxbI72\nGL7QXnc+v94sK04G4y7vERX5J7OrGVzOYdgq6i1uSo0aQK0ysy3nZGCjyDmZre6H69qqOmAzGk20\nPmBxRdV+cVFmNvacDMad13Qvz+zM9wPo74OV7XB6cbwSkJejsHHhZDDO8p+01XM2dL+R1AAPdK93\nMmuW9ra0meCniuNFwIaMkpnlxMmgCfKdtDVgI4OLpW0sM5AGrakx6Wx9Rs1EHTG8A/wUNxPZuPCv\nDBtF56nw5WlwC+n25WmDI4taXbtS8jq9uG0synLR92R9ZWZbzjUDm8A6VGPby4ySwcY1qWlowKKi\nzGzsedKZbVLOSwqkSWdTSbudQdrlbENGk87mPA9tc2HXouQ+oH9FxMqtNvU6s2qedGZbLO8O8HaG\nb3v54ZJi2RztVTWbfcirZmM5aVoykPQ7YAXQB/RGxJ41nnM+cAhpS6djIuKOZsVnI8u3A7zWP++c\nfv9seBKWzhm6HMUG9xnYuGjmNyOArojoqfWgpEOBnSJiZ0l7AV8BFjYxPhtBvstR5LxlJ8DUFcOX\no/jIirKisYmt2T+TNlXFPRxYChARt0raWtKCiPAvoRLlvRzFhoB+Dd1DeGM+nWS01dhetFaZ2ZZr\nds3gekl9wEURsaTq8ZcAj1QcPwpsDzgZlCrn5ShEapV8tDjuI6+9YTzhz5qnmclgn4j4g6RtgOsk\nPRARN1U9p/qbOuxXnKQzKw6XR8TysQ3TJo6ZgjcBvyqODwN+nFM2IN8Jf1YmSV1AV0OvKWNoqaQz\ngFURcXZF2YWki/u3iuMHgP0rm4k8tLT58h9amvN+BvOXwTkHDtbKlgKLrot45qAy47L8tMzQUkkz\ngSkRsVLSLOAg4JNVT7sKOAn4lqSFwHPuLyhf3kNLpzJ8OYeTS4plc/TPr6/MbMs1q5loAXClpIH3\nvCwilkk6ASAiLoqIayQdKulBYDVwbJNis1HkO7S01morOa3A0ktaT2mgmeg+Ur+H2djzDOQmyHdo\nZt6kmf0wS0PH6a+OiDVZZARp9m0wbfehzVzrb49YtUeZcVl+WqaZaDLLe2hm7toCNlQMLd1QlOWi\ng1RBvqo4Pha4uLxwbELL4hdS3jpPheNnpC/0VaT7uaz6mbupbXA8sF1xO74oy8WGuanT+HAGp+Fs\nmFtuTDZRuWYw7tbPT1/iyqaK9e4EbIq1/bCkatvLDRntZzB1QY0ZyAvKisYmNieDcddBjS90SbFM\nNu3rYP1TJj2ZAAALBUlEQVTMwSUp1gMd68qMqDFtG+orM9tyGVWZc5X/kgKSDpbmL0s3HVx2PPXr\nfTxN3v18cZtWlOWi55xUm1la3LqLMrOx55rBuOs5G7r3Jc1+opi0lc2SAnl3gM9YAeeR8UJvt8HG\nXriwIx1u7E1lZmPPyWCc5T1pC/Jemyj3WlnnqXBOR8UM5I58/ttbbpwMmiDfSVu561kO3QcOHncD\nK5aXFIxZS/OkM9ukvNcmmr8Mjj1w6B7Il2aztk/O/+2ttXjSmW2xvJu5+ufDaxkc1ruUnNb2yfu/\nveXGNQObsNJyDlN2H7qhfJ+Xc7BJp55rp4eW2gTXDnywuLkibDYSfztsAutg+BLWnvBnVouTgU1g\nbc/A3cC7iuMdyWtoqVnzuJnIJrCe5bCEwYXelhRl+ch39rflxjUDm8A6u4Y3Ey3qAj5dSjgNynv2\nt+XGycCsZeU8+9ty42RgE5hnIJvVy/MMbMLyDGSzxDOQzYbNQM6HZyBbM7lmYBOWf1mbJfVcO50M\nbEJLCWFgz+ke/7K2ScnJwMzMWmttIklTJN0h6eoaj3VJer54/A5JpzcrLjMza24H8smkZSPnjPD4\nDRFxeBPjMTOzQlNqBpK2Bw4FvgaMVFVx80+LynlJhJxjN2umZtUMzgVOA+aO8HgAe0u6E3gMWBwR\n9zUpNtuEnJdEyDl2s2Yb92Qg6TDgqYi4Q1LXCE+7HdghItZIOgT4HrDLCOc7s+JweUQsH8NwbZic\nl0TIOXazzVdca7saeU0zagZ7A4dLOhSYDsyV9PWIOGrgCRGxsuL+DyR9WVJnRPRUnywizmxCzGZm\n2Sp+JC8fOJZ0xmivaerQUkn7k5qA3lpVvoBUewhJewKXR8TLa7zeQ0ubLOeJWznHbjaWWnU5igCQ\ndAJARFwEvBs4UdJGYA1wRAlxWQ05L4mQc+xmzeZJZ2ZmE1xLTTozM7PW5WRgZmZOBmZm5mRgZmY4\nGZiZGU4GNsF5bSKz+nhoqU1YnnRmlrTqpDOzJvHaRGb1cjORmZm5ZmATWc/Z0L0vUNlMdHapIZm1\nKPcZ2ISW+g06i7WJerw2kU1K9Vw7nQzMzCY4r01kZmZ1cTIwMzMnAzMzczIwMzOcDMzMDCcDMzPD\nycDMzHAyMDMznAzMzAwnAzMzo4nJQNIUSXdIunqEx8+X9FtJd0rarVlxmZlZc2sGJwP3AcMWQ5J0\nKLBTROwMfAD4ShPjahpJXWXHsCVyjj/n2MHxly33+OvRlGQgaXvgUOBrQK3Fkg4HlgJExK3A1pIW\nNCO2JusqO4At1FV2AFugq+wAtlBX2QFsoa6yA9hCXWUHMN6aVTM4FzgN6B/h8ZcAj1QcPwpsP95B\nmZlZMu7JQNJhwFMRcQe1awUvPLXqOJ+1tc3MMjfu+xlI+jTwfmAjMB2YC3wnIo6qeM6FwPKI+FZx\n/ACwf0Q8WXUuJwgzs83QUpvbSNofWBwRb60qPxQ4KSIOlbQQOC8iFjYtMDOzSa6MPZADQNIJABFx\nUURcI+lQSQ8Cq4FjS4jLzGzSymrbSzMzGx9ZzECWtIOkn0i6V9I9krrLjqlekqZLulXSr4rYzyw7\nps0x2qTBVibpd5LuKuL/ednxNErS1pKukHS/pPuKptQsSHpl8d994PZ8Zt/fU4rv7d2SviFpWtkx\nNULSyUXs90g6eZPPzaFmIOlFwIsi4leSZgO3AW+PiPtLDq0ukmZGxBpJ7cDNwMnFfIpsSFoE7AHM\niYjDy46nEZIeBvaIiJ6yY9kckpYCN0TEJcW/oVkR8XzZcTVKUhvwGLBnRDwy2vPLJuklwE3An0XE\neknfBq6JiKUlh1YXSa8Bvgn8BdAL/BD4YEQ8VOv5WdQMIuKJiPhVcX8VcD+wXblR1S8i1hR3pwId\njDzfoiXVMWkwB1nGLWkrYL+IuAQgIjbmmAgKBwAP5ZAIKrQDM4skPJOUzHLxKuDWiFgXEX3ADcA7\nR3pyFsmgkqSXA7sB2fyyltQm6VfAk8CyiPhF2TE1aLRJg60ugOsl/VLS8WUH06AdgaclXSrpdklL\nJM0sO6jNdATwjbKDqFdEPAacDfw38DjwXERcX25UDbkH2E9SZ/Fv5q/YxGTerJJB0UR0BamZZVXZ\n8dQrIvoj4vWk/xF7SXp12THVq4FJg61sn4jYDTgE+HtJ+5UdUAPagd2BL0fE7qTRdh8rN6TGSZoK\nvBX4f2XHUi9J80hL5byc1BIxW9J7Sw2qARHxAPBZYBnwA+AONvGDLptkIKkD+A7wbxHxvbLj2RxF\n9f4nwFvKjqUBewOHF+3u3wTeJOnrJcfUkIj4Q/H3aeBKYM9yI2rIo8CjFbXJK0jJITeHALcV/w9y\ncQDwcEQ8ExEbge+Svg/ZiIhLIuLPI2J/4Dng1yM9N4tkIEnAxcB9EXFe2fE0QtKfSNq6uD8DOJDU\n55GFiPh4ROwQETuSqvk/rpw93uokzZQ0p7g/CzgIuLvcqOoXEU8Aj0japSg6ALi3xJA213tIPyZy\n8ntgoaQZxTXoANLKy9mQtG3x96XAO9hEM10Zk842xz7A+4C7JN1RlP1jRPywxJjq9WJgqaQppOT7\n7Yi4puSYtkTrDz8bagFwZfou0w5cFhHLyg2pYR8GLiuaWh4is0mZRRI+AMiqvyYifi7pCuB20nI6\ntwNfLTeqhl0haT5pNNGHImLFSE/MYmipmZmNryyaiczMbHw5GZiZmZOBmZk5GZiZGU4GZmaGk4GZ\nmZHPPAOzcVdM0OmNiGeb8F4iLVo4E5gFbAXMA7YF7i+WEjBrGicDs0F/CbwZ+MBAgaRjgP9Fmmx3\nBbB/jdcdFxH3S/ogcArwVNXjLwU+EBHXFuf8LfAHYF1xWwusBJ4FeoAnxu4jmdXHk85s0pJ0EvBB\n0oV4QAdptiakX+yXkS7OrwaujYgfbeJ8JwBrI+LrVeVnAD+rSAa3RMQbKh7/R9Jqtrdt+acy2zyu\nGdhktg44lbRkyIMRcbOkA4HHI+JeSW8kLZcOaUmU0+s4Z62VXYeVSTq/4twvB46QNLBUwOURcUH9\nH8NsyzkZ2KQVEV8rNi2ZD5wr6fXANsAvgXsj4kZJOwKzSUv/LpL0tuLluzK4aNltEXESqUbxUUl/\nV/VWC4Abh751vLD1o6TPAd+LiP8c449oVjcnA5vs/hp4P3AGaSGyzwHVi3mtAu4ibYG4hNSs9KOI\n2E/S3IHFv4rdyC6p500lvQL4KfCbomhhsZheZ0Rks9+FTRweWmqTWkR8k9RU9Jpia8AdgcptGQea\neC4lLQH8BWAXeGE70MuK+++T9GtJNxV/j5Z0uqS7irLfFc1Ola6JiP2AfwGOL+5nuU+z5c81A5u0\nio7dt1Qcn0bqP7is+JUO8DrSssWvIA3/3Doi7pFERDwq6TFJ+5BGG50VEV+XdHTx2gBOKpqbPpHe\nQtsyfLTRbkBuW6HaBOOagU1aEfHJiHhDMbLnb0k7QT0AXA0cVpR/CFhflAv4TPHyjuLvZ0gjkgCm\nSpoOTK14G1X9fQNp+8Ee4Kpirfl9gDMl7QssHdtPaVYfDy21SUvSn5LmDbwDmAZ8lLQL2v8EPkK6\naN8OrCt+8U8FjgQWkbaiPLTiXO8ljTZ6gtRh/Blgh+LczwIvI23uchrwKVJH9V+StuBcRNre8qOk\nmshZmxrCajYenAxs0pJ0JOmCfWVE/KbG428jJYJrx/A99wLuAf4euDkiflr1+EuBl0bEzWP1nmb1\ncDIwMzP3GZiZmZOBmZnhZGBmZjgZmJkZTgZmZoaTgZmZ4WRgZmbA/wexgJ+YnQcM3gAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x88af908>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(y_test, y_predictions)\n",
    "plt.xlabel('实际品质',fontproperties=font)\n",
    "plt.ylabel('预测品质',fontproperties=font)\n",
    "plt.title('预测品质与实际品质',fontproperties=font)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "和假设一致，预测品质很少和实际品质完全一致。由于绝大多数训练数据都是一般品质的酒，所以这个模型更适合预测一般质量的酒。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##梯度下降法拟合模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "前面的内容都是通过最小化成本函数来计算参数的：\n",
    "\n",
    "$$\\beta = (X^TX)^{-1}X^TY $$\n",
    "\n",
    "这里$X$是解释变量矩阵，当变量很多（上万个）的时候，$X^TX$计算量会非常大。另外，如果$X^TX$的行列式为0，即奇异矩阵，那么就无法求逆矩阵了。这里我们介绍另一种参数估计的方法，梯度下降法（gradient descent）。拟合的目标并没有变，我们还是用成本函数最小化来进行参数估计。\n",
    "\n",
    "梯度下降法被比喻成一种方法，一个人蒙着眼睛去找从山坡到溪谷最深处的路。他看不到地形图，所以只能沿着最陡峭的方向一步一步往前走。每一步的大小与地势陡峭的程度成正比。如果地势很陡峭，他就走一大步，因为他相信他仍在高出，还没有错过溪谷的最低点。如果地势比较平坦，他就走一小步。这时如果再走大步，可能会与最低点失之交臂。如果真那样，他就需要改变方向，重新朝着溪谷的最低点前进。他就这样一步一步的走啊走，直到有一个点走不动了，因为路是平的了，于是他卸下眼罩，已经到了谷底深处，小龙女在等他。\n",
    "\n",
    "通常，梯度下降算法是用来评估函数的局部最小值的。我们前面用的成本函数如下：\n",
    "\n",
    "$$SS_{res} = \\sum_{i=1}^n (y_i - f(x_i))^2$$\n",
    "\n",
    "可以用梯度下降法来找出成本函数最小的模型参数值。梯度下降法会在每一步走完后，计算对应位置的导数，然后沿着梯度（变化最快的方向）相反的方向前进。总是垂直于等高线。\n",
    "\n",
    "需要注意的是，梯度下降法来找出成本函数的局部最小值。一个三维凸（convex）函数所有点构成的图行像一个碗。碗底就是唯一局部最小值。非凸函数可能有若干个局部最小值，也就是说整个图形看着像是有多个波峰和波谷。梯度下降法只能保证找到的是局部最小值，并非全局最小值。残差平方和构成的成本函数是凸函数，所以梯度下降法可以找到全局最小值。\n",
    "\n",
    "梯度下降法的一个重要超参数是步长（learning rate），用来控制蒙眼人步子的大小，就是下降幅度。如果步长足够小，那么成本函数每次迭代都会缩小，直到梯度下降法找到了最优参数为止。但是，步长缩小的过程中，计算的时间就会不断增加。如果步长太大，这个人可能会重复越过谷底，也就是梯度下降法可能在最优值附近摇摆不定。\n",
    "\n",
    "如果按照每次迭代后用于更新模型参数的训练样本数量划分，有两种梯度下降法。批量梯度下降（Batch gradient descent）每次迭代都用所有训练样本。随机梯度下降（Stochastic gradient descent，SGD）每次迭代都用一个训练样本，这个训练样本是随机选择的。当训练样本较多的时候，随机梯度下降法比批量梯度下降法更快找到最优参数。批量梯度下降法一个训练集只能产生一个结果。而SGD每次运行都会产生不同的结果。SGD也可能找不到最小值，因为升级权重的时候只用一个训练样本。它的近似值通常足够接近最小值，尤其是处理残差平方和这类凸函数的时候。\n",
    "\n",
    "下面我们用scikit-learn的`SGDRegressor`类来计算模型参数。它可以通过优化不同的成本函数来拟合线性模型，默认成本函数为残差平方和。本例中，我们用波士顿住房数据的13个解释变量来预测房屋价格："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.datasets import load_boston\n",
    "from sklearn.linear_model import SGDRegressor\n",
    "from sklearn.cross_validation import cross_val_score\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.cross_validation import train_test_split\n",
    "data = load_boston()\n",
    "X_train, X_test, y_train, y_test = train_test_split(data.data, data.target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "scikit-learn加载数据集的方法很简单。首先我们用`train_test_split`分割训练集和测试集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_scaler = StandardScaler()\n",
    "y_scaler = StandardScaler()\n",
    "X_train = X_scaler.fit_transform(X_train)\n",
    "y_train = y_scaler.fit_transform(y_train)\n",
    "X_test = X_scaler.transform(X_test)\n",
    "y_test = y_scaler.transform(y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "然后用`StandardScaler`做归一化处理，后面会介绍。最后我们用交叉验证方法完成训练和测试："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "交叉验证R方值: [ 0.64102297  0.65659839  0.80237287  0.67294193  0.57322387]\n",
      "交叉验证R方均值: 0.669232006274\n",
      "测试集R方值: 0.787333341357\n"
     ]
    }
   ],
   "source": [
    "regressor = SGDRegressor(loss='squared_loss')\n",
    "scores = cross_val_score(regressor, X_train, y_train, cv=5)\n",
    "print('交叉验证R方值:', scores)\n",
    "print('交叉验证R方均值:', np.mean(scores))\n",
    "regressor.fit_transform(X_train, y_train)\n",
    "print('测试集R方值:', regressor.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "##总结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "本章我们介绍了三类线性回归模型。首先，通过匹萨价格预测的例子介绍了一元线性回归，一个解释变量和一个响应变量的线性拟合。然后，我们讨论了多元线性回归，具有更一般形式的若干解释变量和一个响应变量的问题。最后，我们讨论了多项式回归，一种特殊的多元线性模型，体系了解释变量和响应变量的非线性特征。这三种模型都是一般线性模型的具体形式，在第4章，从线性回归到逻辑回归（Logistic Regression）。\n",
    "\n",
    "我们将残差平方差最小化为目标来估计模型参数。首先，通过解析方法求解，我们介绍了梯度下降法，一种可以有效估计带许多解释变量的模型参数的优化方法。这章里的案例都很简单，很容易建模。下一章，我们介绍处理不同类型的解释变量的方法，包括分类数据、文字、图像。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
